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Abstract 

We calculate the partition functions of supersymmetric gauge theories on S 5 , which 
acquire non-perturbative contributions from instanton loops wrapping its Hopf fiber. The 
instantons on the CP 2 base equivariantly localize to 3 fixed points of SU(3) Cartans. 
Using our results, we study the index of the 6d (2, 0) theory. We first study the partition 
functions of maximal SYM with AD(E) gauge groups, which agree with the gravity dual 
indices on AdS-j in the large N limits. We also show that the most general partition 
function of the Abelian theory agrees with the index for a free 6d tensor supermultiplet. 
We explain possible ambiguities in our 5d approach, role of maximal SUSY, and the 
Wilson loops on S 5 dual to M2-brane Wilson surfaces in AdSj. 
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1 Introduction and discussions 

The discovery of M-theory as a non-perturbative type IIA string theory [1] was perhaps one of 
the most dramatic events in the history of string theory. The quantum mechanical emergence 
of a new (circle) direction of M-theory, often showing its Kaluza-Klein states as quantum bound 
states of DO-branes, is the essence of this phenomenon. The type IIA string theory description 
and a more geometric M-theory description often exhibit very different pictures of various 
phenomena at weak/strong couplings. 

Such weak/strong-coupling dual descriptions of type IIA/M-theory have been studied in 
various lOd/lld manifolds. The classic studies of |T] start from the type IIA string theory on 
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R 9,1 . Similar studies can be made on various curved backgrounds, also including interesting Eu- 
clidean spaces. Among others, studies by Gopakumar and Vafa [2] suggest that the topological 
string partition function on IR 4 xCY3 with a gravi-photon deformation has a dual interpretation 
at strong coupling as a Witten index counting 5d BPS states on M 4 x S 1 , with the emerging 
M-theory circle at strong coupling interpreted as a Euclidean time direction. 

The idea that Euclidean type IIA partition functions of various sorts can count states of 
M-theory was applied to the study of M5-branes from 5d gauge theories living on D4-branes, 
the type IIA counterpart of M5's. More generally, there have been studies on the 6d (2, 0) 
superconformal theories on curved backgrounds including a circle factor, from the dimensionally 
reduced 5d gauge theories. It was often the Yang-Mills instanton solitons in 5d which carried 
information on the quantum emergence of an extra circle direction and the 6d theory. The 
earliest study of this sort is [3], with various other ingredients like DLCQ and so on. 

At least apparently, 5d Yang-Mills theories are non-renormalizable. So one may wonder 
whether one can do any consistent quantum calculation at all with these 5d theories. One 
attitude towards this issue is to restrict one's interest to certain supersymmetric observables. 
There are examples in which nonrenormalizable low energy effective descriptions of UV complete 
theories can be used to calculate a selected set of observables in a consistent manner. The 
basic idea is that extra symmetries can make nonrenormalizable theories more predictive than 
generic expectations. Especially, quantum fluctuations for supersymmetric observables often 
experience huge boson/fermion cancelations so that a priori wild quantum fluctuations of non- 
renormalizable theories can be consistently controlled. There are many examples of this. Just 
to mention some, firstly Nekrasov's instanton partition function on I 4 x S 1 [H El E] can be 
calculated for non-renormalizable 5d gauge theories. See also [7] for the usage of this partition 
function in 5d maximal SYM on IR 4 x S 1 to study the 6d (2, 0) theory on R 4 x T 2 . Also, studies 
of black hole partition function from 4d M = 2 supergravity [8] have been made. Often, the 
underlying framework turns out to be the 'localization' of the supersymmetric path integrals, 
which states that a path integral enjoying certain fermionic symmetry is secretly a Gaussian 
path integral around a set of supersymmetric saddle points. UV quantum fluctuations come into 
a good control for the Gaussian path integrals. Also, supersymmetric observables calculated 
this way can often be directly understood from string theory. 

Although this attitude is what we shall basically assume throughout this paper, we also 
extensively consider the 5d maximal SYM as a special case, for which our study might have 
implications on recent discussions on possible UV finiteness of this theory [9]. See also [TO] . 

Our strategy above was recently applied to the study of the index of 6d (2, 0) theory [TT1 
[T2] counting local BPS operators, which can also be regarded as a supersymmetric partition 
function on S 5 x S 1 with various twists along the Euclidean time circle S 1 . The key idea is 
to study the supersymmetric partition function of a suitable SYM theory on S 5 , which can be 
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regarded as a naive dimensional reduction of the 6d theory along the circle [13]. See also [H] for 
the on/off-shell formulations of general 5d M = 1 gauge theory on S 5 , and [TS1 EES! [EH [TS] for 
earlier and following works which study closely related subjects^ In this paper, we continue to 
study the 6d index from gauge theories on S 5 , with emphasis on the full microscopic derivation 
of the nonperturbative part of the S 5 partition function. 

The setting and results of [13] will be reviewed and extended in section 2.1. Just to explain 
salient aspects which are relevant here, the QFT on S 5 in [13] came with two dimensionless real 
parameters. One is the ratio of the Yang-Mills gauge coupling g\ M and the 5-sphere radius r, 
which we call (3, and another is the ratio m of r _1 and the mass for the adjoint hypermultiplet. 
These are related to two chemical potentials of the 6d index [13]. The path integral of this theory 
localizes to a Gaussian one around a set of saddle points. The saddle points are characterized 
as follows. Firstly, the saddle points are characterized by self-dual Yang-Mills instantons on 
the CP 2 base of S 5 in Hopf fibration. The Hopf fiber is picked by selecting a supercharge that 
one uses to calculate the path integral, which is either chosen arbitrarily (for the calculation 
of unrefined indices of [13]) or uniquely fixed (when one considers the most general index with 
all chemical potentials turned on). As the saddle point configuration is homogeneous along 
the Hopf fiber, this is an instanton string or instanton loop winding the Hopf fiber circle of 
S 5 . On top of this, there is also a Hermitian matrix which completes the parametrization of 
the saddle point space, given by the value of an adjoint scalar in the 5d theory. So one first 
has to evaluate the determinant of quadratic fluctuations with fixed scalar VEV in various 
instanton sectors, and then integrate over the Hermitian matrix to get the partition function. 
The perturbative measure of the matrix integral at zero instanton number was calculated in 
[T5l [16j HBJ. In some special cases, nonperturbative contributions are suggested in [13] with 
various consistency checks. 

One main goal of this paper is to provide a derivation of the non-perturbative partition 
function, with a highly nontrivial generalization by including all allowed chemical potentials in 
the 6d index. 

The instantons appearing in our localization are self-dual (meaning not anti-self-dual) on 
CP 2 , in the convention that the Kahler 2-form of CP 2 is anti-self-dual [13]. There exist 
ADHM like constructions for the instanton moduli space on CP 2 [22] for certain gauge groups, 
even though we shall not use this formalism in our studies. The study of this paper is rather 
based on the so-called equivariant localization and index theorems, which we review in section 
2. The equivariant deformation utilizes the isometries on the instanton moduli space and refines 
the partition function. The deformation in our case will use the U(l) 2 Cartans of the 577(3) 
isometry of CP 2 , and comes with two parameters which we call e\, 62- The deformation lifts the 

1 Similar S 5 partition function was also studied for 5d theories with different gauge groups and matters |19j , 
with 5d UV supcrconformal fixed points. The S 4 x S 1 index for the same class of theories was studied in [20] . 
Also, the S 5 x S* 1 index was studied from an appropriate supersymmetric gauge theory on CP 2 x S 1 [2T] . 
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position moduli of the instantons on CP 2 . With this, one can use equivariant generalizations 
of Atiyah-Singer index theorems to calculate the BPS modes on S 5 which contribute to the 
S 5 partition function. ei,e 2 play the role of chemical potentials conjugate to U{1) 2 charges. 
In particular, one can use the equivariant localization to calculate such indices, which states 
that the index can be calculated by studying the indices around the fixed points of the U(l) 2 
isometries. See section 2.2 for the details. As there exist 3 fixed points on CP 2 , the determinant 
factorizes into 3 factors inside a matrix integral. Each part is obtained from an instanton 
calculus on M 4 with a U(l) fiber, i.e. the Hopf fiber of S 5 . We should note that, for such a 
localization of instantons to happen (so that we can luckily use the instanton calculus on M 4 ), 
it is important for these instantons to be in the self-dual sector, excluding the Kahler 2-form 
J. This is because an anti-self-dual instanton configuration proportional to J will never be 
localized to fixed points, as J is SU(3) invariant. 

On flat space, the equivariant parameters 61,62 are introduced as a sort of IR regulator 
for the noncompact IR 4 moduli space of instantons coming from their positions, meaning that 
61,62 — > limit is divergent [I]. As CP 2 is compact, in our case one can of course turn them 
off and obtain the undeformed partition functions and 6d indices, which reduce properly to 
the results obtaineed previously [T5J [16l [13]. However, it is also very useful to keep the 61,62 
parameters as extra chemical potentials in the 6d index for the rotations on S 5 . Together with 
the other two parameters (3 ~ - 01 and m of the theory, they provide 4 chemical potentials in 
total, which form the maximal set admitted by the 6d (2, 0) superconformal index. 

After a general derivation of the perturbative/non-perturbative partition function on S 5 
with all four parameters kept, we make closer studies of this quantity as the 6d superconformal 
index. To study the 6d index from our results, there are two technical challenges that one has 
to overcome, which we partially achieve in this paper in some special cases. 

The first challenge is that our expression naturally takes the form of a weak-coupling expan- 
sion at small coupling <C 1, because our result is organized as an infinite series coming from 

47T 2 fc 

various instanton sectors weighted by a factor like e ^ < 1 for k instantons. On the other 
hand, the index nature of the partition function is visible by an expansion with fugacities, most 
importantly with e _/3 which is conjugate to an energy-like conserved charge on S 5 x IR. The ex- 
pansion with e _/3 <C 1 is in the strong-coupling regime from 5d viewpoint. To get to the strong 
coupling regime from our instanton series, one has to sum over all instantons and re-expand the 
exact expression at large (3. So far, we managed to do so in two special cases, in which we have a 
good technical control over the instanton series. Firstly, with 5d maximal SYM, corresponding 
in 6d to tuning some chemical potentials, the instanton series is significantly simplified so that 
we recast it using the Dedekind eta function. It is easy to re-expand this function using its 
modular property and study the 6d index. Secondly, the Abelian instanton sum is considerably 
simpler than the general non- Abelian one, in that it does not depend on the Hermitian matrix 
for the scalar expectation value. In this case, one can directly obtain the 6d index by cleverly 
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mimicking the techniques explored by Gopakumar-Vafa [2], rewriting the partition function 
into an integral similar to Schwinger's proper time integral, and then re-expanding it at strong 
coupling. The resulting strong-coupling expansion can be compared with the Abelian 6d (2, 0) 
index directly computed using the free tensor supermultiplets [12]. They completely agree with 
each other. Although this calculation is trivial in 6d, it is a nontrivial example which illustrates 
that our 5d approach is properly working. 

Another technical challenge of getting the 6d index from the 5d partition function is that 
our general expression for the partition function takes the form of a matrix integral with a 
nontrivial measure. This is sometimes called the 'Coulomb branch localization' [231 12"%]. as the 
matrix coming from a nonzero scalar is formally analogous to the Coulomb branch of QFT 
vacuum on flat space. In this paper, we mainly study the Abelian case in which the integral is 
trivial, or the case with maximal SUSY in which the integral is simplified. 

At least for U(N) gauge group, which admits a Fayet-Iliopoulos deformation which lifts 
the Coulomb branch, there could be another type of localization for gauge theories, in which 
the saddle points are given by a discrete sum over points on the 'Higgs branch' of the QFT. 
This was recently shown to be possible for supersymmetric partition functions on S 2 [231 El] . 
Also, at the level of supersymmetric quantum mechanics, similar idea has been applied to 
provide alternative derivations of the instanton partition function on IR 4 x S* 1 or R 4 [7] and 
vortex partition function on IR 2 x S 1 or IR 2 [25]. More 'standard' derivations of these partition 
functions for instantons [I] and vortices [26] can be regarded as 'Coulomb branch localizations' 
in that a complex mechanics variable in the vector supermultiplet remains unlocalized. The 
'Higgs branch localization' of [25] sums over discrete saddle points. Of course the details of 
Coulomb vs. Higgs branch localizations depend on situations. However, if such an alternative 
localization is possible on S 5 without any matrix integral, perhaps the index structure of the 
S 5 partition function could be made more manifest! 2 ] 

Overcoming the two technical challenges in full generality, if possible, would be useful not 
just for studying the 6d (2, 0) theory but also for studying various (1, 0) superconformal theories 
on S 5 x S 1 from 5d gauge theories on S 5 . For instance, the 5d Sp(N) gauge theory with Nf = 8 
fundamental hypermultiplets and one hypermultiplet in anti-symmetric representation is a circle 
reduction of the 6d (1, 0) SCFT with E s global symmetry. The 6d theory lives on a stack of N 
M5-branes near the Horava-Witten 9-plane, whose circle reduction yields N D4-branes near an 
08-plane and Nf = 8 D8-branes. It should be interesting to study this model and check, for 
instance, the enhanced E s global symmetry which is not visible in the 5d theory. Similar E n 
type enhancements of global symmetries of 5d SCFT were studied in [20] for n < 6 using the 
index [12] on S 4 x S 1 . 

The remaining part of the paper is organized as follows. In section 2, we first explain the 5d 
2 We thank Jaume Gomis and Davide Gaiotto for suggesting and emphasizing this possibility. 
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field theory setting for calculating the 6d (2, 0) superconformal index. Then using equivariant 
indices, we compute the perturbative and instantonic contributions to the 5 5 partition function. 
In section 3, we provide three applications of our results to study the 6d index. In section 3.1, 
we show that the Abelian index obtained from our 5d instanton calculation perfectly agrees 
with the index for the 6d free tensor supermultiplet. In section 3.2, we study a special unrefined 
index for the ADE gauge theory keeping only one chemical potential, which is computed from 
the maximal SYM on 5 5 . The A n and D n cases can be fully studied, while the E n cases are 
studied only up to perturbative level. The A n and D n indices completely agree with the large 
N gravity dual indices on AdSj. We briefly comment on the cases with non-ADE gauge groups, 
having in mind the interpretation as twisted indices of 6d ADE theories. Section 3.3 explains 
the expectation values of Wilson loops on 5 5 and compare them with the dual M2-brane Wilson 
surfaces in AdS-j. Section 4 explains possible small ambiguities in our 5d approach and how 
we suggest to fix them. Two appendices explain the technical details of instanton calculus and 
the structure of our off-shell QFT from supergravity. 

As we were finalizing the preparation of this paper, we encountered [271 12H] which partly 
overlap with our work. The formula in [27] with three factors of topological string partition 
functions is similar to our three factors of localized instanton contributions on CP 2 . Also, our 
section 3.1 for the Abelian 6d index overlaps with [27]. [28] derived the perturbative partition 
function that we also study as a part of our result. 

2 The partition function on S* 5 
2.1 QFT set-up and geometries 

In this subsection, we review and generalize the results on 5d SYM on (squashed) 5 5 , from 
which we shall calculate the 5d partition function. 

The 5d gauge theory on 5 5 of our interest was presented in [13J. The suggestion was that 
the partition function of 6d (2,0) on 5 5 x 5 1 , or the superconformal index [12] counting local 
operators of this theory on M 6 , can be computed from the 5d SYM theory which one can regard 
as its dimensional reduction to 5 5 . Let us start by explaining the 6d (2,0) superconformal 
symmetry. The 05p(6,2|4) symmetry of this theory has 6 Cartans in the bosonic subgroup 
50(6,2) x Sp{4) ~ 50(6,2) x 50(5). The Cartans of 50(5) are called R u R 2 , which rotate 
the two orthogonal 2-planes of internal M 5 in a way that spinors carry ±| charges. The angular 
momenta of 50(6) C 50(6, 2) are called ji, J2, J3, which rotate 5 5 in a way that they rotate the 
three 2-planes of M 6 with half-integral charges for spinors. The energy (or conformal dimension 
for local operators) E is the 50(2) C 50(6,2) charge, made dimensionless by multiplying r. 
The states counted by our index saturate the BPS bound E > 2{R\ + R 2 ) + ji + j 2 + J3 [12]. 
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See the beginning of our section 3 for the details of this 6d index. 

In [13], the 6d index with two chemical potentials (3, (3m conjugate to E — Rl + R2 and 

„2 

f3m = Ri — i?2 wa s considered. j3 = ^xm. is related to the 5d Yang-Mils coupling, m is r 
times the mass of the adjoint hypermultiplet in 5d M = 1 SYM theory. The derivation of the 
5d theory in [13] relied on the strategy of first reducing the Abelian 6d theory on S 5 x S 1 to 
SYM on S 5 , and then trying a non- Abelian completion while securing the desired supergroup 
SC/(4|l)i See pj] for the details. 

The localization of the partition function of this theory was explained in [13]. By picking a 
particular Hermitian SUSY for the calculation, one first has the saddle point given as follows. 
One has a constant scalar among 5 adjoint scalars in the theory, which should be exactly 
integrated over after the localized path integral for other modes is done. Also, the saddle point 
admits nontrivial instanton configurations on the CP 2 base of S 5 , being homogeneous on the 
S 1 Hopf fiber. The Hopf fiber on the round S 5 is picked by selecting a SUSY among 8 (in 
[131 [T6| [T3]) or among 16 (in [H]). One may either understand this choice as being arbitrary, 
or unique if one has in mind the squashing of S 5 we explain below. See also [T8] . 

One can make a step further to include more chemical potentials of the 6d index into the 
parameters of the 5d action. As we explain in section 3, the remaining two chemical potentials 
are for the two angular momenta j\ — j2, ji — jz in 577(3) G SO(Q) which acts on the CP 2 
base of S 5 . The natural way of incorporating the two parameters is by suitably squashing 
S 5 . One can write down a gauge theory on this space with 2 Hermitian supercharges. Gauge 
theories on a class of squashed S 5 were studied in [18]. In this paper, we view the squashing of 
S n realized as dimensionally reducted chemical potentials for angular momenta on S n , in the 
index on S n x S 1 . This has been emphasized, for instance, in [20] which discusses the squashed 
S 4 partition function as a reduction of an index on S A x S 1 . This agrees with the computation 
on the squashed S* 4 of 'ellipsoid type' in [29] . 

In this paper, rather then discussing the gauge theory on squashed S 5 and its superalgebra 
there in full detail, we shall mostly work by deriving only some aspects of this squashed theory 
that we need in our calculation. Most information that we need shall be derivable by considering 
Abelian 5d theories on the squahsed sphere, as the localization calculation often relies on the 
quadratic part of the action with a straightforward non- Abelian generalization (at least when 
saddle point studies are concerned). The quantum fluctuations are controlled solely by the 
structure of the Q 2 superalgebra which we use in the localization calculus, which is again 
obtained from Abelian theory and then straightforwardly generalized. The on-shell Abelian 
theory can easily be obtained by reducing the Abelian 6d tensor theory and dualizing it in 5d. 
For the 5d non- Abelian theories, we mostly assume that a non- Abelian generalization of the 5d 
Abelian theory exists, with same superalgebra. For instance, this has been rigorously checked 

3 Whcn m = ±i, symmetry enhances to SU(A\2) with 16 real SUSY, realizing maximal SYM on S 5 . 
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to be true for gauge theories on S 5 without squashing. Especially, our detailed calculation 
would be carried out around the fixed points of isometries on the squashed S 5 , for which we use 
the equivariant localization technique. For this, it suffices to know the local QFT's on M 4 with 
a i7(l) Hopf fibration. However, in appendix B we set up the off-shell supergravity derivation 
of the QFT on squashed S 5 , to derived some facts we need. This formulation should admit one 
to derive the full QFT straightforwardly. 

The {7(1 ) 3 rotation on C 3 (which contains our S 5 given by \Zx\ 2 + \Z 2 \ 2 + \Z 3 \ 2 = 1) acts as 

{Z h Z 2 , Z 3 ) -+ (Z ie ia \Z 2 e™\Z 3 e ia *) . (2.1) 

The overall {7(1) rotation of all Zj shifts the Hopf fiber coordinate, which is wrapped by the 
instanton loops and thus acts trivially on the saddle points. As we shall introduce chemical 
potentials for the SU (3) acting on CP 2 only, we impose the constraint a\ + a 2 + a 3 = 0. One 
may take the following two combinations 

ei = ai - a 3 , e 2 = a 2 - a 3 , (2.2) 

which will play the role of Omega deformation parameters around a fixed point. On CP 2 , there 
are 3 fixed points of U(l) 2 on CP 2 , which are 

Z = (Z 1} Z 2 ,Z 3 ) = (1,0,0) , (0,1,0), (0,0,1) (2.3) 

up to a phase rotation which is not in CP 2 . Around each fixed point, CP 2 is approximated by 
M 4 with Omega deformation parameters given by 



(1,0,0) 
(0,1,0) 
(0,0,1) 



(a 2 - ax, a 3 - a ± ) = (e 2 - ex, -ex) 
(a 3 - a 2 , ax - a 2 ) = (— e 2 , ex - e 2 ) 

(ax - a 3 ,a 2 - a 3 ) = (ex,e 2 ) . (2.4) 



The instanton partition function will take the form of a product of three instanton partition 
functions on twisted M 4 x S 1 , where S 1 is the Hopf fiber circle. This is similar to the localization 
of instantons on S* 4 around its north and south poles |30j . 

Being the chemical potentials of SU(3) rotations, 01,02,03 or e 1; e 2 can be understood as 
twisting the 6d geometry. Let us consider this geometry and part of the bosonic action which 
should be important to us. The U(l) 2 C SU(3) chemical potentials Oj = (a, 6, c) subject to 
a + b + c = are encoded in the 6d geometry as 



ds* = r 



dnf + nf I d0iH -dr ) + dn 2 + n 2 I d(f) 2 -\ -dr J + dn 2 3 + n 2 ( d(p 3 -\ -dr 



+dr 2 



(2.5) 

where ni + ri^ + n^ = 1. We replaced ctj above by iai with real a^, as the 6d chemical potentials 
we introduce are real, which deform the action in a complex way. The above complex metric 
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is nothing but reflecting a complex deformation of our QFT. To KK reduce on the circle 
t ~ t + 2nri, one rewrites the metric as 



dsl = r 2 



r 2 . 2 , ,2 . {Of^hf 

dn { +71^ + 2 2 



dT+ irrfad^ 
1-nfaj 



We are also interested in the metric near the three fixed points, at which one of the rij's are 1 
and other two 0. For example, let us consider the fixed point with ri\ — 1. The results for the 
other fixed points are similar, with the roles of a\, 03 changed. The metric locally looks like 

2 



ds 2 6 



ds 2 (R 4 ) + 1 



1-a? 



H l-ai)[dr + ^) , (2.7) 



where the factor of R is provided by n2,ris,(p2, 



»3- 



Now we consider the deformation of the superalgebra by this squashing. Before squashing, 
the Hermitian supercharge in S77(4|l) that we choose in S 5 x S 1 satisfies the following algebra 
(up to varios gauge transformations when Q 2 acts on gauge non-invariant objects) 

&„E- 2{Rl + R * )+jl+j2+j3 . (2.8) 

r 

E ~ — -gp is the translation along S 1 with dimension of mass. The effect of twisting ( 12. 5 p is to 
covariantize all time derivatives in the 6d (Abelian) equations of motion and SUSY by 

d d ia,i d ai . 

— — or E^E ji . 2.9 

or or r ofa r 

As we further twist the time translation by chemical potentials (3, (3m conjugate to E — f?1 + R2 , 
f?1 ~ fl2 , there is an extra shift of energy by Rl + R2 — m(Ri — Rq). After this shift, reducing the 
6d theory to S 5 will force us to restrict to E ~ — = 0, leaving us with the 5d algebra 

3 

- Q 2 ~ ^-(Ri + R 2 ) + m(Ri ~R 2 ) + - ^(l+ajji . (2.10) 
Zr v . 

i=i 

We shall use this Q on S 5 to localize the path integral. 

One can also consider the reduced 5d action obtained from the 6d (2, 0) theory along the 
r circle in the 6d geometry ( 12. 5 p or (12. 7p . One would first obtain constant warp factors on 
the 5d metric from the (1 — a 2 ) factor of the r circle, which will affect various terms in the 5d 
SYM (or in the Abelian case, supersymmetric Maxwell-like theory). Secondly, the line element 
dr + % ™^2 at the last term of (12. 7p induces a new term in the 5d action when one reduces the 6d 
(2, 0) theory. This is because the (imaginary) 1-form C ~ '^t 1 appearing in the line element 
for the M-theory circle can be regarded as a Ramond-Ramond 1-form in type IIA perspective. 
Namely, the Wess-Zumino coupling between 'the RR bulk field' C M and the '5d worldvolume 
field strength' F of the form j s5 C A F A F is induced from the twisting of the M-theory metric. 
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Of course, our metric (j2.7p is not the full M-theory metric and the above argument in the honest 
sense applies to the lid metric pull-backed to 6d worldvolume of M5-branes. However, taking 
Abelian 6d (2, 0) theory on any curved space and reducing it on a circle factor, this heuristic 
argument can be shown to hold rigorously in the 6d context [31]. Based on these ideas, we 
capture the aspects of our gauge theories on S 5 or twisted M. 4 x S* 1 coming from squashing. We 
discuss the basic setting of localization and the classical aspects of this calculus here, leaving 
the quadratic quantum fluctuations to the next subsection. 

With a symmetry Q which squares to a bosonic symmetry of the path integral, the corre- 
sponding supersymmetric path integral admits a deformation of the action by 

S^S + t{Q,V} (2.11) 

for any V which is invariant under Q 2 , without changing the result of the integral. One simple 
choice of V is V ~ Q, as 

[Q 2 ,V] ~ [Q 2 ,Q] = . (2.12) 

The last equation holds from the Jacobi identity. Thus, the deformation of the action which 
one uses to localize the path integral is tQ 2 with t — > oo. Therefore, in this setting, one has to 
calculate the determinant of Q 2 in the saddle point background. 

In [HI [T5J [lEl [13], the saddle points of the localization calculation were given by Yang-Mils 
self-dual instanons on the CP 2 base of Hopf fibration, with a covariantly constant scalar 0, 
before one turns on the squashing parameters. After one turns on 61,62, breaking £77(3) into 
U(l) 2 in the Q-exact deformation tQ 2 , the saddle points are further restricted by requiring 
that the deformed Q 2 still vanishes at the saddle point. From the right hand side of the SUSY 
algebra (I2.10p . the instantons should be invariant under 

3 

53(l + fli)ji (2.13) 
i=i 

among others. With all a« = 0, the component ji + 32 +33 left the instanton loop configuration 
invariant as the loop winds the Hopf fiber. With generic satisfying a + b + c = 0, the 
instanton's position moduli get further constrained. On the triangle formed by n 2 ,n 2 ,n 2 , one 
finds that they have to be localized on one of the three fixed points of U(l) 2 described in the 
previous subsection, with two of the rij's being zero. As the nonzero field strength is localized 
to three points, the covariantly constant scalar <fi at the saddle point is simply taken to be a 
constant scalar away from the fixed point, similar to the results on £ 4 |30j . 

We now decide the value of classical action at the saddle points. We consider the sector 
with gauge fields and scalar mass terms, as these determine the classical action which straight- 
forwardly generalizes to non- Abelian case. Firstly, in the notation of [31], the 'type IIA dilaton' 
$ and 'RR gauge field' C M are defined from the 6d metric as 

dsl = e-**dsl + e**{dr + e^C^) 2 . (2.14) 
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Rewriting (12 .7D in this form, one obtains 



C 



irac 



a 2 



(1 -a')* 

Or if one considers the full geometry, one would obtain 



ds 2 



I -a 1 



(2.15) 



r 2 (l -n?a?)5 



Our 5d action, which is the quadratic part of their DBI action, is 



^9ym 



d 5 x^ e-*F^F>" 



e~*C A F A F 



(2.16) 



(2.17) 



^9ym 

Compared to [31], a factor —i is multiplied to the WZ term as we consider a Euclidean theory, 
with Lorentzian and Euclidean times are related by t = —ir. Decomposing 5d indices to 
i — 1, 2, 3, 4 on M 4 and 5 for 1; one finds the following M 4 part of the gauge field action: 



d 5 x 



2 n 



.p. .p. . 

11 11 



8g£ M (l - a 5 



-e^F^Fu 



(2.18) 



where z, j indices on M 4 are contracted with 5ij. It is the saddle point value of this action under 
self-dual gauge fields on local R 4 which will be important for the calculation of our partition 
function. Imposing the self duality condition, one obtains 

4tt 2 4tt 2 



S 



1 



racpi 



^ XFijFij " ^ * & M (1 + a) ~ ftl + a) ■ 



(2.19) 



The factor of multiplying the standard instanton action ^ at the first fixed point has to 
be replaced by ^zg an d f° r the second and third fixed point, respectively. Therefore, the 
instanton expansions around the three fixed points have to be made in 



e m+^H) (< 1) for i = 1,2,3 
at the three fixed points labeled by % — 1, 2, 3. 



(2.20) 



In [131 [16] . it was also important to obtain the value of classical action for nonzero con- 
stant scalar in the vector multiplet (and also an auxliliary field D proportional to 0) in the 
localization calculation. On the round S 5 , the classical action at the saddle point was 



e~ SoW with S 



2n 2 tr\ 2 



(2.21) 



Here (3 = ^p 1 and A = r<f). This is obtained by plugging in the saddle point value and 
D 3 = -0, in the notation of [13], into their quadratic classical action terms. Note that the 
off-shell action and SUSY were important for obtaining the correct coefficient, as the above 
saddle point value for D 3 is different from its on-shell value D 3 = —-<f>. To get the correct 
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generalization of this Gaussian measure after squashing, one should redo the off-shell analysis 
of SYM on the squashed S 5 . 

In appendix B, we study the off-shell supergravity method [321 [T3] to find that the Gaussian 
measure is generalized to 

o„2 tr \2 

e~ So with S = - , , (2.22) 

/0(l + a)(l + 6)(l + c) V ; 

with our squashing with a + b + c = 0. We also provide a nontrivial support that the coefficient 
of the Gaussian should be given as f !2.22p . This can be seen from a careful study of the 6d 
Abelian index that we study in section 3.1. Note that the only place this Gaussian coefficient 
appears in the Abelian case is the overall prefactor of the partition function. After a careful 
regularization, we confirm that the above factor is compatible with the known 6d index. 



2.2 Equivariant indices and the partition function 

Now with the classical result at the saddle points, we calculate the determinant of the operator 
Q 2 from the Gaussian flugcuations in t — > oo limit. Without squashing, this determinant in the 
zero instanton sector was calculated either by brutal calculation with S 5 spherical harmonics 
[T3] or more efficiently by using appropriate index theorems [151 EE] • 

We use equivariant index theorems to calculate the determinant on squashed S 5 , in general 
instanton sector. As in [IB] , we view S 5 as a U(l) Hopf fibration over CP 2 and use appropriate 
Atiyah-Singer index theorems on CP 2 . The index theorems used in [151 US] can De generalized 
to include more equivariant parameters ei,e2, as we explain below. The relevant index for the 
5d M = 1 vector supermultiplet is that for the so-called self-dual complex. The hypermultiplet 
index is that for the Dirac complex. See, for instance, [33] for a summary. Both indices will be 
related to the index for the Dolbeault operator By on CP 2 , covariantized by appropriate vector 
bundles appearing on the right hand side of (12.101) . In paticular, V includes the U(l) bundle 
from the Hopf fiber. The index for the Dolbeault complex is given by 

ind(«9 y ) = J Td(TCP 2 ) A e Cliy) , (2.23) 

where Td(TCP 2 ) is the Todd class for the holomorphic tangent bundle of CP 2 . We shall explain 
various factors above (in our equivariant version) later. Here let us just leave a comment that, 
on CP 2 with a Hopf fibration, one includes C\ ~ t J with the Kahler 2-form on CP 2 for the 
£7(1) bundle, when one considers the index in a sector with U(l) charge t on CP 2 . The values 
of t for various fields can be read off from the right hand side of ( 12.101) . namely the value of 
|(i?i + R 2 ) + (ji + j2 + jz)- ji + H + jz takes integer values. So for the vector multiplet, 
with Ri = i?2 = 0, t is valued in integers. For the hypermultiplet, there are two scalars with 
(Ri, R2) = (1,0), (0, 1). So t should be shifted by a half-integer, say by |. 
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It will turn out that the full nonperturbative forms of these indices in instanton backgrounds 
are most easily derived after turning on nonzero ei, e 2 - In particular, the equivariant localization 
theorem that we shall employ admits us to use the ADHM instanton calculus on flat space [6], 
with a simple generalization. To address the equivariant indices of our interest, one first replaces 
all the cohomologies used in the index by the equivariant ones with nonzero e\, e 2 . Namely, the 
exterior derivative d is replaced by 

D = d + z v (2.24) 
with a Killing vector v on CP 2 . In our case, v is given by 

Odd 

v ~ a i^r + a 2^r + a 3^r • ( 2 - 25 ) 
d0i d(p2 d(p 3 

The closed forms for d are no longer closed with D. So one has to find deformed differential 
forms closed in D to get the equivariant indices. Among others, the Kahler 2-form J of CP 2 
can be deformed to an equivariantly closed form as follows. In the homogeneous coordinates 
Zi in the previous subsection, one obtains 

J = l -dZi A dZi . (2.26) 

d is generalized to 

D = d + i v , v = -2a\{iZ\d\ - iZSx) - 2a 2 (iZ 2 d 2 - iZ 2 d 2 ) - 2a 3 (iZ 3 d 3 - iZ 3 d 3 ) . (2.27) 
One obtains 

DJ = -a 1 (Z 1 dZ 1 + Z 1 dZ 1 ) + --- = -d/j, (with /i = a x \Z x \ 2 + a 2 \Z 2 \ 2 + a 3 \Z 3 \ 2 ) . (2.28) 

So one finds the following equivariant generalization J e of J: 

J t = J + fx, DJ e = . (2.29) 

J e replaces J in c\(V) in the equivariant indices. As explained in the previous subsection, our 
convention in this paper is to use imaginary Omega deformations. So we replace all a^s to ia^s 
in the above formulae. The equivariant version of Todd class is given as follows. The ordinary 
Todd class of a complex n dimensional space is given using the n Chern roots Ai, A2, ■ ■ • , A n by 

Td = n^- ( 2 -30) 

i=l 

On M 2n , with deformations e\, e 2 , ■ ■ ■ , the i'th Chern root is shifted by an addition of a 0-form 
i€i in our convention. Thus, the 0-form component of the equivariant Todd class on M 4 is 

Td e = . . 1 ][) V — - , 2.31 
(e 4£ i - l)(e" 2 - 1) v ' 

which will be all we need to calculate our indices. 



13 



The equivariant versions of various indices mentioned above have the same forms, with 
ordinary characteristic classes used to define them replaced by their equivariant versions. The 
extra e parameters play the role of chemical potentials for j\ — j 3 , j 2 — J3 appearing on the right 
hand side of Q 2 in (12.101) . These indices will all take the form of 

ind = ^2 ni e- w ' . (2.32) 

i 

Here i labels various eigenvalues of Q 2 for BPS modes captured by the index, Wi is the cor- 
responding eigenvalue, the integers are the index degeneracies of BPS modes with given 
eigenvalue. These data are related to the 1-loop determinant of Q 2 by 

det = Y[w- n >. (2.33) 

i 

The eigenvalue Wi will contain various continous parameters like m, (3, as well as the saddle 
point value A of the vector multiplet scalar. In the instantonic sectors, one would have extra 
martix </> in the adjoint representation of the instanton gauge symmetry. After obtaining this 1- 
loop determinant, one should suitably integrate (ft and A to get the final (squashed) S 5 partition 
function. 

The final ingredient that we need to calculate our indices is the so-called equivariant local- 
ization. It states that the integral of an equivariantly closed form a € is given by [34] 




(2.34) 



where p labels the fixed points x p of the isometry v on CP 2 . a e (x p ) is the 0-form value of 
a e at the p'th fixed point, and x( x p) 1S the 0-form value of the equivariant Euler class there. 
As we already explained, there are three fixed points of v. The local values of a t are those 
on M 4 , which we explained for important characteristic classes above. The Euler class of M 4 
with parameters e\, e% is x = (^1X^2)- Now with equivariant localizaiton, calculation of a e (x p ) 
almost boils down to that of various indices in the instanton background on local M 4 x S 1 near 
the fixed points, except that the circle is nontrivially fibered so that e Cl for the U(l) bundle 
has to be multiplied. From (I2.28p . (I2.29p . the 0-form value of J e at the i'th saddle point (with 
\Zi\ = 1) is iaj. This provides a multiplicative factor 

e a(v) e «M ( 2 _ 35 ) 

at the i'th saddle point, when the U(l) charge of the BPS modes is t. For hypermultiplets, t is 
replaced by t + | with an integer t. 

Apart from the above factor from U(l) fibration, the remaining factor of a e (x p ) is almost 
the instanton index on M 4 x S* 1 [El [33] with minor changes. At an M 4 around a fixed point with 
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Omega background given by €1,62, the self-dual/Dirac indices that we need are related to the 
Dolbeault index (12^5]) by 



I _|_ e i(ei+e 2 ) _ n+t2 e im _|_ e ~im 

indsD = g ind(dy) , ind Dirac = e l ^~ ind(cV) . (2.36) 

The factor e im + e~ tm comes from m(Ri — R 2 ) on the right hand side of (I2.10p . for the two 
hypermultiplet scalars with {Ri^R^) = (1,0), (0,1). The Dolbeault index for instantons on 
M 4 x S 1 was studied in detail in [BJ. At a fixed point with e-y, e 2 , one obtains [BJ 

Ch(£)Td(TC 2 ) ^ CHE) ^ _ ±- _ 1} , (2.37) 

where the second factor divides the 0-form value of the Todd class by the Euler class, as required 
in f !2.34p . Ch(S) is the equivariant Chern character for the so-called 'universal bundle' [BJ, which 
depends on A for the gauge symmetry of the theory, for the instanton gauge symmetry, among 
others. Ch(£) is given for the classical gauge groups with various representations in the second 
reference of [6j, where ADHM formalisms of instantons are known. Here we explain it mostly 
for U(N), leaving the detailed formulae for other classical gauge groups SO(N), Sp(N) in 
appendix A.l. The result for the adjoint representation of U(N) is given by 

Jffi-l) = (^iff-l) - 6 "'^ MOrf*> + cc.)+(l-e-<«)(l-e-<«)tr^(e«* 

(2.38) 

in the k instanton sector, k and k denote the fundamental and anti-fundamental representa- 
tions of the instanton gauge symmetry U(k), while N, N denote those of U(N). This form 
is to be used for calculating the determinant for the vector multiplet, as well as an adjoint 
hypermultiplet which is our main interest. For the hypermultiplet in general representation R 
of the gauge group, the systematic method for constructing Ch(£) was explained in the second 
reference of [BJ, with many examples. The second and third terms on the right hand side of 
(12.381) come from the zero modes in the instanton background, which depend sensitively on the 
choice of R [6j. However, the first term is simply the perturbative contribution to the index, 
which for general R is replaced by 

tr i? ( elA ) 

(2.39) 



(e iei - l)(e" 2 - 1) 

Of course (I2.39P also works for exceptional groups. We explain the structure of this perturbative 
part in more detail in section 2.3. 

Briefly commenting on the results for other classical gauge groups, various modifications 
are made as follows. Firstly, for the G = SO(N), Sp(N) gauge theory, the k instanton gauge 
symmetry is Gk = Sp(k),0(k), respectively. For the vector or hypermultiplet fields in adjoint 
representations, the characters for adj N and adj fc appearing in (I2.38P are replaced by those of the 
adjoint representation of G and appropriate representations of Gk, as explained in appendices 
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A.l and A. 2. The fundamental representations at the second term of (12.381) are replaced by 
fundamental representations of G, Gf.. For hypermultiplets in general representation R of G, 
the instanton part of Ch(S) again changes a lot due to different instanton zero mode structures. 
We again refer to [6] for more details on general R. 

Collecting (|2T34j) . (12T35D . (12T36D . (12371) . (12T381) . one can write down the full U(N) equiv- 
ariant indices for various fields in U(N) adjoint representation, in the background of self-dual 
instantons on S 5 . Firstly, let us specialize to a contribution from the fixed point on CP 2 with 
\Z 3 \ = 1, with Omega background €\ = a — c, e 2 = b — c. Other two contributions can be easily 
obtained by permuting a, b, c. The vector multiplet index is given by 



X _|_ e *(ei+<:2) 



cfuHi-Ce 



422 = E ^ {1+C)A T, w ■ i ( 2 - 4 o) 

vector 2 Z-^i [ (1 - e lei )(l - e le2 ) 



; 11+12. 



(tr N (e* x )tr- k (e^) + c.c.) + (1 - e~^)(l - e- 2 )ch adjfc (0) 



where t is an integer for ji + j 2 + j$ charge along the U(l) fiber, r 1 plays the role of its 
chemical potential. Similarly, the hypermultiplet index in adjoint representation is given by 

e i(m+|(l+c)) +e i(m+|(l+c)) ^ ,^,„w„r ch a ^Je iX ' 



T U{N) 6V 2V "+ e> ^ ST Jt(l+c)/r ^dj N {d ) , , 

'hyper 3 2^ [ h _ e Uy) h _ £ ie 2 ) V AL > 

i=-oo v > y > 

^ (tr^e^tr^) + c.c.) + (1 - e^l - e- 2 )ch adjfc (0) 



_ • El +E9 

— e 



All 1 + c factors in these expressions come from the chern class for the U(l) bundle given by 
(12.351) . Also, for the hypermultiplet, the factor |(1 + c) which shifts the mass m comes from an 
appropriate half-integral shifts of the U(l) charge appearing in (12.101) . as explained after (12.231) . 
The above form can be achieved by suitably shifting the integer t in the summation, which we 
find to be convenient. See appendices A.l and A. 2 for more indices with other classical groups. 

At this point, we should carefully explain various terms above more concretely. The funda- 
mental characters and U (N) adjoint characters appearing in the expressions are standard: 

N N k 

C h adJJV = ei(Ai " Aj) > ch ^ = $> Ul = ( ch *)* > ch * = = ( ch ^)* > ( 2 - 42 ) 

i,j=l 1=1 1=1 

apart from the subtle property of the perturbative part including chadj^ that we explain in 
detail in section 3.30 As for the hypermultiplet, the adjoint character of U(k) in the last term 
is also given in the standard way as ch ad j fc = X)j j=i e 1 ^ 1- ^^. For the vector multiplet part, 
the precise definition of the last term is given after expanding ei , e 2 dependent factors as 

k k 

- (1 - e" 4£1 )(l - e-^ch^e**) -)• - + (e~ iei + e~ lt2 - e~ i{ei+e2) ) ^ e'^ _ * j) . 

i^j i,j=i 

(2.43) 



4 The perturbative index with the above character includes the N saddle point values Aj themselves which 
we do not Gaussian integrate. We should exclude them from the index. 
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The reason why the contribution with / = J is excluded becomes clear when one understands 
the meaning of all 4 terms. The first term with negative coefficient comes from the U(k) gauge 
invariance constaint in the nonzero <pj background: as the latter manifesetly preserves only 
U(l) k , the remainders appear as constraints with I ^ J. The next two terms come from the 
bosonic generators of the ADHM data in U(k) adjoint representation. (Bi-fundamental ADHM 
data are encoded in the term with fundamental characters above.) The last term comes from 
the complex ADHM constaints imposed on the generators. 

We now consider the determinant of the form (12.331) . To motivate the final expression, 
let us first comment on some schematic structures. As the eigenvalue t run over integers, the 
determinant contains an infinite product over t, with eigenvalues taking the form of 

— — — — + (expressions containing e±, €2, m, <p, A) . (2.44) 

This infinite product after regularization becomes a sine function, meaning that going from the 
equivariant index to the determinant is essentially like taking Plethystic exponential, apart from 
the factor (1 +c) multiplying t. The last factor is an effect coming from the circle fibration over 
M 4 . The arguments of the sine functions in the determinant are all divided by 1 + c compared 
to the determinants for instantons on M 4 x S 1 , absorbing all factors of r to other parameters. 
Another difference is that the hypermultiplet has its mass factor effectively shifted to 3 ^ , 
coming from the shift of t eigenvalue by t + | from its nonzero R-charge in the algebra. Apart 
from these, the resulting measure is very similar to the instanton determinant on R 4 x S 1 so that 
we can essentially borrow the techniques for the instanton calculus there [HG3I6]. Remembering 
the above difference and taking the product over t, the determinant from a fixed point with 
\Z$\ = 1 takes the form of 

^pert^inst ' (2-45) 

The perturbative part Z peTt coming from the first terms of (12.401) and fl 2 . 4 1 j) will be explained in 
more detail in section 3.3, as one should carefully expand the denominators into infinite series. 
The remaining terms in the indices are finite series apart from the t summation, so one obtains 
a finite number of sine factors in the determinant. The result is 



7 (3) 
^inst 



k\ 



n acpi 
2tt 

.1=1 



1 \ 3(l+c) , , . 3(l+c) 

Sin 7T — SU1 7T- 



1 sm7 r J , , — ± smn VI + 

1 = 1 1 = 1 l + c 1 + c 



X 



smiT :Llj T- : — ± sm7r r— sm7r 



, . . 3(l+c) , , . . 3(l+c) 



U TT 0111 11 l+c 0111/1 l+c 0111 " l+c 

1 + C 11 sin7T 0/J ~ £l shlTT 07 - 7 "' 2 • 4>lJ+m+^^-+e+ . r j >IJ+m +^±^-e + 
I+J I, J &m " l+c olli " l+c sin 7T— t— 1 sin it— 2 - 



1+c 



where <pu = <pi — (fij, e± = The overall factor of (1 + c)~ k comes from a mismatch of the 
factors of (l + c) pulled out from ( 12.44ft . coming from the lack of the I = J terms in the fisrt 
factor on the second line of (I2.46p . At this stage, we already integrated over the 0/ variable for 
the U(k) gauge symmetry. This expression takes a form very close to the instanton partition 
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function on t 4 x S 1 . The rules for the contour and the classification of poles are explained in 
[I]. Using €i = a — c, e 2 = b — c and a + b + c = 0, one can rewrite the above contour integral 
formula as 



7 (3) 
; inst 



(l + c)~ fc 

k\ 



2tt 



.1=1 



k N ■ -ri "i — 9 

sm vr 7- 2 — sin it 



nn 

7=1 i=l 



1+c 



1+c 



sm 7r y : , — ± sm 7r— 



(2.47) 



1+c 



X 



sm 7T 



fru+3e 
1+c 



sm7T- 



1+c 
_(f>ij+m—^+a 
1+c 



sm 7T- 



A/j+to-|+6 
1+c 



©r r— a+c • 0r j—b+c 

sm 7r , , sm ir— — — 



1+c 



1+c 



sin 7T - 



4>u+m+i 



1+c 



Sin 71"- 



frrj+m+f +3c 
1+c 



Again, the results for other gauge groups are given in appendix A.l. 



The integration over the k eigenvalues <j)j can be explicitly done for the U(N) theory, yielding 
a simple formula for the residues. The correct contour for this integral is known from [4], also 
reviewed in detail in [TJ. The poles contributing to this integral are classified for the U(N) 
theory by the A-colored Young diagrams with k boxes. The A-colored Young diagram is given 
by the set of N Young diagrams Y = (Y\, Y%, • ■ ■ , Yn) with the total number of boxes being k. 
The residue at this pole for U(N) group takes the following form |35j : 

N ■ By+m+5e+£l_ e+ . £ m _MI+£)_ £+ 
, , SIM— -— 2 Sin7T— 7— 2 

4 3) = n n ! +c El] . e^- + i+c — (2-48) 

is i ,cv S m 7T TTT- Sin 7T \, + 

i,3=l s£Yi 1+c 1+c 

with 

Eij = Xi - Xj - e x hi{s) + e 2 (vj + 1) . (2.49) 

Here, s labels the boxes in the z'th Young diagram Yi. hi(s) is the distance from the box s to 
the edge on the right side of Yj that one reaches by moving horizontally. Vj(s) is the distance 
from s to the edge on the bottom side of Yj that one reaches by moving vertically. See [351 E] 
for more explanations with examples. Collecting all residues, and also summing over different 
instanton numbers, the full instanton partition function at this fixed point is given by 



Z 



(3) 



inst 



k=0 Y; \Y[ 



£ 



.(3) 
'Y 



(2.50) 



with the k instanton weight e "(i+c) explained in section 2.1. 



Similar to the above contribution zj^Lz., 



from the third fixed point \Z 3 \ = 1, there are 



contributions Z^Z^ 



and Z^l+Z^ from the first and second fixed points on 



IF . These two 



pert '"inst 

port -"inst 

are obtained from the third one by replacing a, b, c by b, c, a and c, a, b, respectively: namely, a, b, 
c are playing the special roles in the formulae for the first, second, third fixed points, respectively. 
The integration over the U(N) eigenvalues Aj has to be made with this determinant and the 
classical contribution (I2.22p in the measure. The S 5 partition function takes the following form: 



Z(P,m,e 1 ,e 2 ) = 



\dX t 



-£(i4^+6) (i4c) Z^ ■ Z^ z^ ■ z^ z^ 

pert inst pert inst pert inst 



(2.51) 



18 



All factors generically depend on /3,m, 61,62 nontrivially, which we suppressed in the above 
expression for simplicity. The perturbative determinants are carefully considered in section 3.3. 
There we shall be careful about the overall normalization issue, which will eventually make the 
overall factor above meaningful. This factor can be understood as a division by the order of 
U(N) Weyl group. An important feature that one should remember is that the round sphere 
limit 61,62 — > is smooth. This is true only after combining all factors from the three fixed 
points. In particular, there are many ways of taking the limit of the two parameters. The 
existence of the smooth limit implies that same result is obtained irrespective of the relative 
rate at which the two parameters are sent to zero. 



2.3 More on perturbative determinant 

Let us consider the perturbative determinant in more detail. In this subsection, we consider 
general gauge group G. Contrary to the instanton part of the index, the perturbative part is 
an infinite series in e lei , e 162 at each fixed point. At each value of t, the index summed over 
all three fixed points again becomes a finite series, related to the fact that the e\, 62 — > limit 
of the total index is smooth. There are two possible ways of organizing the net perturbative 
contribution. One is to identify the finite series for given t after summing over three indices, 
and then calculating the determinant. Another way is to first expand each perturbative index 
in a definite order of e t€1 , e 1,62 , calculating the determinant of each and then multiplying the 
three. Although the latter apparently looks inefficient, it actually provides a useful expression 
of Zpert so that we explain both in turn. 

Collecting the perturbative determinants from the previous subsection, the net perturbative 
equivariant index for vector multiplet at given value of t is given by 

7-pert,+ 1 rpert,— 

Ccw(^ 2 ) = -e il r D ° M 2 D ° M ch adj (e^) (2.52) 

„-it(ei+e 2 )/3 p it(2ei-e 2 )/3 ii(2e 2 -ei)/3 
rpert,+ / \ _ c I ~ 1 \_ 

DoM ^ i, 2) ^ _ eiei ^ _ gie ^ t ^ _ ei (e 2 _ ei ))(i - e -fci) ^ (1 _ e - fc a)(l - e^i"^)) 

e -it(e 1 +e 2 )/3 e i(e 1 +e 2 ) e it(2ei-e 2 )/3gi(e 2 -2ei) e it(2e 2 -ei)/3 e i(ei-2e 2 ) 



1 - e iei )(l - e ie2 ) (1 - e i ( e2 " e i))(l - e~ iei ) (1 - e^ £2 )(l - e^ 61-62 )) 

pert,+ ; 
Dol, — t' 



7-pert,H 
( Dol,t 



is essentially the perturbatve part of the Dolbeault index. Similarly, for the hypermul- 
tiplet in representation R, one obtains at given t (after a little rearrangement) 

i(m+|) rpert,+ . j( m+ |) j-pert,- 

C = ^ - — -^^r ^- ch «( e ") • ( 2 - 53 ) 

We inserted a = £2 ~ 2£1 , b = £l ~ 2£2 , c = — ei ~^ €2 in these expressions. 
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If we expand I^if 1 , which determines all the above indices, the sum of three factors yields 
a simple finite series. In terms of a, b, c, one finds 

{ e it/r Y^ +q+r=t e i(pa+qb+rc) _ jy>+1+r=t e i( p (I +a ) +g (I+&) +r (I +c )) ^ > 
W "° (t 1-2) 

e it/r jy+q+r=-(W) e -i(pa+qb+rc) = ^P+9+r=-(^)g-i((>H)(fM)+(?H)(i-t*)+(r+l)(fjc)) ^ < _ 3 ) 

(2~54) 

As for I^il~(a-, b, c) = ^fjof^tl - a ' ~~ ~ c )i one finds 

{ e it/r ^p+g+r=t-3 e i( pa+qh+rc ) _ ^ p+ q+ r =t~3 g i ( (j>4-1) ( ^rfa) +(q+1) ( ^H>) +(r+l) ( i-H;) ) ^ > ^ 
^ (t =1,2) . 

e it/r ^P+<?+r=-t e _j( pa+g 6 +r . c ) _ ^ p +g+r=-f e i(p(I+a)+g(i4*)+r(i+c)) ^ < 

(2.55) 

Note that I^ oi l with £ > is simply the degeneracy for rank t holomorphic polynomials in 
C 3 . The €i, e 2 — > limit agrees with the perturbative index studied in [T5] . 



The spectra of /doH 1 * 1 are completely mapping to each other with a sign flip on the eigenvalues. 
Since the remaining factor ch a dj is real, the contributions of two factors in the numerator of the 
vector multiplet index (I2.52p yield same factors. The same is true for the hypermultiplet if the 
representation R is real, but not in general. 

With this information, one can immediately write down the determinant as the following 
formal product (redefining raj to be new ajjj: 

oo 

det v = [ (p(l + a) + q(l + b)+r(l + c) + a(X)^ ((p+l)(l + a) + (g+l)(l + 6) + (r + l)(l+c) + a(A) 

aSroot p,q,r=0 

(2.57) 

One has remember that the index theorem is simply counting the BPS modes, so whether we 
are Gaussian integrating them or not is what we should decide. In particular, the mode in the 
first factor with p = q = r = 0is not to be kept when a = 0, as they are constant scalar modes 
A which we should integrate over exactly at the final stage. For the hypermultiplet, one obtains 

det H = f[ h?(l+a)+g(l+6)+r(l+c)+m+-+^(A) ) (p(l+a)+q(l+b)+r(l+c)+m+- -n(\)) 



/iGwcight p,q,r=Q 



x ((p+l)(l + a) + (g+l)(l + 6) + (r + l)(l + c)-m-^+//(A) 

_ i 

(p+l)(l + a) + (g+l)(l + 6) + (r + l)(l + c)-m-~MA)J ' • (2.58) 



5 By abusing the notion of roots, in this subsection we definite a to run over all weights of generators, 
including Cartans. Also, the scalar expectataion value A ins 5d QFT is real, so that one will have to replace 
A — > i\ to use these determinants in field theory. 



20 



When the representation R is real, we can replace — //(A) in the product by /i(A), after which 
one obtains 

00 / 3 A" 1 

det H = [ [p{l + a) + q{l + b)+r{l + c)+m+- +/x(A) J 

AtGweight p,<j,r=0 ^ 

x ^(p+l)(l+ a ) + ( g +l)(l+6) + (r+l)(l+c)-m-|+/x(A)^ . (2.59) 

For the theory with an adjoint hypermultiplet, which is our main interest, one can use the last 
formula. All infinite products have to be regularized, after which one obtains a generalization 
of the triple sine function. To naturally do so, it should be desirable to rewrite the above p, q, r 
prodcut (which is democratic in the a, b, c) as an infinite product over t and further to a finite 
product in two integers, like the restricted sums which appear in (12.541) . (I2.55p . The t product 
could be regularized using zeta functions. One should obtain same pre-factors as those studied 
in the e u e 2 limit [321 [321 OB] . 

When a = b = c = 0, only p + q + r = t appears in the product. The perturbative determinant 
cam be written as (we changed A — > iX to go to QFT convention) 

00 00 
det y = [] 11^ + + 3 + «(A)) ™ = [I «(A) [I (H »(A)) t2+2 

a t=0 a t=\ 

OO , l, ( f + lK t+ 2) l. V + W+2) 

det H = I U+ - + m + i/i(X)) (t+- + m-i^(X)) (2.60) 

(i t=o ^ ' ^ ' 

1 (t+l)(t+2) 1 (t+l)(t+2) 

/ 3 \ 2 2 / 3 \ 2 2 

x lt+- — m + ifx(X) J (t+- — m — ifi(X) 



dety was calculated in |T5]. When the hypermultiplet representation is real, one obtains 

OO x (t+l)(t+2) (t+l)(t+2) 

det ff = JJJJ^ + - + m + i//(A)J " ^ + - - m + z>(A) J " . (2.61) 

This agrees with the result of [13], with the parameter A in that paper related to m here by 
A = m + |. It was also noted that, at m = ±| when we expect to have SUSY enhancement 
on S 15 , detydetjj- experiences further cancelation so that the net measure becomes the product 
of sinh functions [13]. This form will be used in section 3.2 to study a particular class of 6d 
indices. 

It would also be useful later to have a different form of the above product, obtained by 
directly expanding each fixed point index and recasting them into a Plethystic-like exponential. 
For instance, we expand all denominators of the indices in power series of e i£1 , e t€2 assuming 
Im ei > Im €2 > 0. By expanding in different orders, one would be able to get similar expressions 
with the roles of a, b, c exchanged. After some calculation, one obtains the following product 
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form for det 



v ■ 



oo oo 



dety = ] \^(l + a)t+je l + ke 2 + a(X)y^(l + a)t + (j + l)e 1 + (k + l)e 2 + a(X)y (2.62) 

a t=—ooj,k=0 



x ( (l+b)t+je 1 + k(e 1 -€ 2 )+a(X)J ^(l+b)t+(j+l)e 1 +(k+l)(€ 1 -e 2 )+a(X) 
I (l + c)t+(j + l)e 2 +k(ei-e 2 ) + a(X)) 2 f (l + c)t+je 2 + (k + l)(e 1 -e 2 ) + a(X) 



The variable t in the product is the same t we used for the index, j, k are for the infinite series 
expansions of various denominators in the indices. We emphasize again here that, in the above 
product over t,j, k, the first factor on the first line and the first factor on the second line with 
t = j = k = has to be excluded in the product when a = 0, corresponding to the Cartans. 

One can factor out all the (1 + a)t, (1 + b)t, (1 + c)t terms in the product, and then take a 
product over t to obtain an expression which becomes and infinite product over j, k with sine 
functions. After rearrangements, and pulling out the formally divergent prefactors, one obtains 
(with given a) 



oo oo 

nn 

i=l j,fc=0 



;i+a)(l+6) 



1+c 



*n 



j,k=0 



?r(l + c) 



X 



1+a 1+a l+o l+o 



n 
n 



stoT MM^ sin 

l+c 1+c 



1/2 



1— e !+° 



1 — e !+ a 



P_,- jgi-4*(gi-gaH^ 

1-e i+ b 



o_.- (j+i)n-Kfe4-i)(c 1 -€ 2 H-^ \ n 1 / 2 

1-e 1+ 5 



A/" PE' 
A/" PE' 



1 + e x + a e !+ 



e l+ e 2 



1-e 



2ri - 



2ni 



1+c 



1-e 



2?ri 



je 2 4<fcfl)( ei _ e2 >fA 



l+c 



,27ri 



2tU- 



2(l-e 2 ™^ IM-e^ifi ) 2( l-e zm TTF )( l-e zm " ) 2 ( 1 -e z ™^ )( l-e Z7 ™^+ 



,2tu- 



1 + e j e"""^ 

+ 



2ni — 
l+c I g 1+c 



2tu 



2-Tri- 



,27Tj- 



■ b+c-2q 



e !+" 



Isin7r+- - sin7r-, . 

1+a 1+a 



— e 7rll + a + (a, &, c — > b, c, a) + (a, 6, c — > c, a, 6) 



(2.63) 



where 

oo 

u v =M a n 

i,fc=0 



n 



t=l 



l+a)(l+6) 
1+c 



;i+a)(i+&) 

2iri(l + c) 



(2.64) 



exp 



.(2j + l)e 1 + (2A; + l)e 2 ,(2j + l)e 1 +(2fc+l)(e 1 -e 2 ) . (2j + l)e 2 + (2fc + l)(e 1 -e 2 )' 

— 7TZ ; 7TZ h 7TZ 



1 + a 



with 



1 + 6 
if a is a root 



1 + c 



i 



— r if a = 0, for a Cartan 

(1+a) 2 (1+6) 2 



(2.65) 
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Here, PE denotes the P let hystic- like exponential, regarding all expression like 
as 'chemical potentials.' The primes in the product only applies when a 



0, corresponding to 

states in the Cartan. The prime in this case excludes the contributions from 'zero eigenvalues' 
for t = j = k = 0, and the primes in PE also exclude the corresponding —1 's in the exponential, 
which will yield 0. Also, the structure of the prefactor M needs explanation, the factor 
^t(i+c)^ on ^ ne fist line inside the j, k product appears by extracting the factors from 
the eigenvalues in (12.621) with t = 0, as they are providing the leading linear factor x in 
sin(7ra;) = vrxf^^Q (l + |). However, note that one does not have such sine factors for j = 
k = 0: for this, there are no 1 + a and 1 + 6 factors to be pulled out into N '. This explains 
the division by (1 + 0)2(1 + 6)2 in M. a . Later, when one considers a gauge theory with one 
adjoint hypermultiplet, all the prefactors M from vector and hypermultiplets cancel out except 
for this Ai a part, as there are no zero eigenvalues in the hypermultiplet to be removed from 
the product. The exclusion of —1 in the exponent of PE for the Cartan components can be 
expressed more manifestly by writing 



det v = Af v PE 



,- 6+c-2q 



1 + a + e 



j b+c — 2a 
1 + a 



sin^^ sin^f- 2 - 

1+a 1+a 



+ (a, 6, c — > 6, c, a) + (a, b, c — > c, a, b) + 1 



(2.66) 



In the expansion with Im e\ > Im > that we have assumed, the +1 at the end should be 
divided into +| + 1 and each +| has to combine with the first and second term in the exponent 
to make the expansions free of the constant term — |'s (i.e. zero eigenvalues). For the hyper- 
multiplet, very similar expansion can be made without worrying about the zero eigenvalues, 
even for t — j — k — 0. After a similar algebra, one obtains 



det H = N^ 1 P.E 



■ 2m + 3(l + a) 



+ e" 



■ 2m + 3(l + a) 
1 (T+a) 



isinvr^sinTr^ 

1+a 1+a 



e i+a _|_ ( a? 6, c — 6, c, a) + (a, b, c — >■ c, a, b) 



(2.67) 



where Mh takes exactly the same form as Mv above, except for the absence of Ai a factor. 

We also explain the form of the net perturbative determinant for the theory with one 
adjoint hypermultiplet, combining detv and detH, which will be our main interest in the next 
section. Firstly, the prefactors Afy, Mh almost cancel but not exactly, due to the absence of 
t = j = k = zero modes in the vector multiplet. The net pre-factor is 



AT, 



1 



1 



a 2 



(1 + 6): 



(2.68) 



where r is the rank of the gauge group. The prefactor in this expression is asymmetic in the 
exchange of a, 6, c, simply because we are expanding Z peTt in an asymmetric way. Such an 
exchange symmetry should be there, although implicit in the expressions. The remaining part 
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can also be organized as 



PE' 



= M H 
with 



. 2m+3(l + a) _.■ 2m+3(l + a) „■ b+c-2a 

_g m (1 + a) _g m (1 + a) -(-e 77 * 1 + a -|_ e _ 



E ■ Q (A) 
e "is 4- (a, 6, c permutation) 

(2.69) 



1 sin 71-^ sinvrf- 21 

1+a 1+a 



exp 



Y \ /(P> a ' 6 > c ) Z_> e 1+a + (a, 6, c -»• 6, c, a) + (a, 6, c -+ c, a, 6) " - 1' 



f(p,a,b, c) 



p7r(m-i+6) . pir(rn— ^+c) 

sin — h— 2 — - sin — 2 — - 



1+a 



1+a 



pirla—b) ■ pTr(a-c) 

sin ^ — n — - sin — 



(2.70) 



1+a 1+a 

The "—1" subtraction from the absent zero modes is to be made only when a = 0. In the 
expansion prescription of denominators that we are advocating, this —1 has to combine with 
the third term in the exponent to make it finite. The last structure, together with the prefactor 
(12.681) will be important for comparing our Abelian partition function with the 6d Abelian 
index in section 3.1. 



3 Applications 

In this section, we shall study our 5 5 partition function as the 6d (2, 0) superconformal index 
on 5 5 x 5 1 . Before studying specific cases, let us start by explaining the general structure of 
this index. 

The 6d (2,0) theory, with 05p(6,2|4) symmetry, has 5*0(6,2) x 5*0(5) bosonic symmetry 
with 6 Cartans Ri, R 2 , ji, j 2 , J3, E as explained in section 2.1. Among the 32 supercharges, 
16 Poincare supercharges Q's have E = |, (Ri,R 2 ) = (±~,±~) as a 4-component 50(5) 
spinor, and finally (j'1,,7'2,.73) = (±-|, with the constraint that the product of three 

eigenvalues is negative. Conformal supercharges 5's are Hermitian conjugate to O/s, so they 
have conjugate charge contents to O/s. We choose Qf \ = Ql'l _ = Q and its conjugate 
^?i,hJ3 = = "^j an d consider an index which counts states (or operators) annihilated by 

them [12]. The superconformal algbera yields 

{Q, S}~E- 2(R 1 + R 2 ) - Ui + k + h) ■ (3.1) 

The states counted by the index that we shall define in a moment saturate the following BPS 
energy bound, given by their five Cartan charges: 

E>2(R 1 + R 2 ) + (j 1 +j 2 + j 3 ) . (3.2) 

So for these BPS states, we only have 5 independent charges labeling them. Among these, 
only four of them can be used to define a Witten index [12]. This is because the Witten index 
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demands that only the combinations of these 5 independent charges which commute with Q, S 
be used to weight the states. We may take the following 4 linear combinations: 



Ri + R 2 

E , Rx-Rz, ji - j 3 , ]2-]3 



The superconformal index is defined by 

J(/3,m,e 1 ,e 2 ) = Tr [(-D^'W^g-/^- 



) e /3™(Ri -Ra) g-7i Gh -33 ) g-72 (32 - 33 ) 



(3.3) 



(3.4) 



where the trace is taken over the Hilbert space. /3' is a regulator which does not appear in the 
index. As for the chemical potentials 71, 72, one may introduce a, b, c (subject to the constraint 
a + b + c = 0) and rewrite 

-/3{aj 1 +bj2+cj 3 ) 



e -71 (jl - J3 ) g-72 Q?2 -J3 ) 



(3.5) 



inside the trace, relating 71 = /3a, 72 = (3b. The parameters a, 6, c are the squashing parameters 
in the 5d theory that we have been discussing in section 2. Also, the (3,m are interpreted on 
S 5 as the square of gauge coupling g\ M = 2tt(3 and the hypermultiplet mass [13] . 



3.1 Index for the Abelian 6d (2,0) theory 

The index for Abelian 6d (2, 0) theory can be calculated directly in 6d, as the theory is free. 
In this sense, this theory is somewhat trivial. However, it provides a highly nontrivial testing 
ground for tour 5d approach, and especially to concretely see how it works. 

Apparently, note that the dimensionally reduced Abelian 5d theory is also free. In particular, 
this means that the perturbative part of the partition function is given by a simple Gaussian 
matrix integral as there are no perturbative interactions at all. However, the instanton part is 
often subtle even for the Abelian theories, as one is often required to consider singular small 
instantons in the quantum theory to do the correct physics [36]. Strictly speaking, these small 
instantons are beyond the rigorous scope of our QFT in a narrow sense. However, there have 
been various proposals to regularize the singular instantons (including the Abelian instantons 
that we discuss here) by introducing certain UV regulators. For instance, in the U(N) theory, 
noncommutative deformation was shown to admit such a regularization at short distance |37j . 
making even Abelian instantons to be regular QFT solitons. We think these small instantons 
could be providing (maybe a small amount of) UV completion data beyond the 5d QFT to make 
the latter useful to study 6d SOFT. See section 4 for the summary of all such ambiguities in our 
approach. Anyway, these small instantons make even the Abelian 5d theory to be nontrivial 
enough to reproduce the 6d physics. 

We also note that studies of the Abelian theories in 5d / 6d are recently made on T 5 and T 6 
[38| [39] . In particular, the approach of [39] for studying the Abelian instantons on spatial T 4 
may be useful even for studying our non- Abelian small instantons on CP 2 with a U(l) fiber. 
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In our convention for the chemical potential, the 6d Abelian index is given in terms of the 
following 'letter index' 



(e^ m + e~P m ) - e" 2/3 (e /3a + + e Pc ) + e" 3/3 
/ {p, m, a, b, c) - ^ _ ^j^yjfi _ ^T^jyi _ e -/3(i+c)) ' ' ^ 

One can quickly understand various terms and factors as follows. The free 6d tensor supermul- 
tiplet contains 5 real scalars in 5*0(5) vector, 16 fermions ipf^jf^ in 5*0(5) spinor and 50(6) 
chiral spinor (same chirality as the Poincare SUSY), and a self-dual 3-form tensor. Among 
them, the BPS fields with respect to Q, 5 defined above are given as follows: two complex 
scalars which have (i?i, R 2 ) charges (1, 0) and (0, 1), three fermions V*- + +) ^+ - +j ^+ + and 
none from the 3-form field. The first two terms in the numerator of (13.61) come from two BPS 
scalars, and the next three terms from the three BPS fermions. The three factors in the de- 
nominator come from the multiplications of three BPS derivatives, which carry scale dimension 
E = 1 and (j\,j2,3z) charges (1,0,0), (0,1,0), (0,0,1), respectively. So one is only left to 
understand the last term of the numerator. This comes from a BPS constraint from fermion 
equation of motion, as one component of their Dirac equation turns out to be constructible 
from BPS fermions and derivatives only. Note that is anti-chiral, and one of its (ji, ji, jz) 
charges is (=, §, a)- This, together with its scale dimension E — ~ and R\ — R 2 — saturates 
the BPS bound (13. 2J) . The full Abelian index is the Plethystic exponential of /: 



PE[f] = exp 



^2-f(pP,m,a,b, c) 



P =i F 



(3.7) 



00 00 



'\ — g2+ni+n 2 +n 3( An.i-l^ri2 ( An3'j^ _ ^2+711+722+^3^1^2-1^3^-^ _ ^2+m+n2+n3^1^2^n3-l\ 

m=0 m=0 n3=0 



nnn^ 



y^ +ni+n2+n3 crc2 n2 C3 3 )(i - y- i ^ +rti+n2+n3 crc2 n2 c 3 n3 )(i - q z+nMn <xrQ 3 ) 



where q = e , y = e /3m , Q = (e /3a , e /3b , e /3c ) for % — 1, 2, 3 (satisfying C1C2C3 = !)■ 

To the above /, which just counts BPS excitations, one can multiply the 'index Casimir 
energy' factor [13], which is the summations of all zero point charges multiplied by the chemical 
potentials. With the chemical potentials /3, f3m, ftcii (i = 1,2,3), this factor is an exponential 
of /3 times some quantity which does not depend on fi. For instance, in the free theory, the 
calculation can be done as follows, by introducing a UV regulator j3' as usual for zero point 
quantities: 



lim tr,. 



' modes 
'-f0 



[-If (e - - m(R 1 - R 2 ) + adi} e ^'(^-^-(^-^)+^)' 



(3.8) 

where the regulating exponential is introduced in the most general form, subject to the con- 
straint that it has to commute with Q, S which are symmetries of our index. The linear 
appearances of m, labels different kinds of zero point charges. Once we do this calculation 
for the free theory, eliminating or renormalizing the divergences proportional to the inverse 
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powers of (3', one obtains The 'Casimir energy' factor 

>h — hr — rn\S -I- A 2 \ 

(3.9) 



e _/3e ° = exp 



(3 f 2abc + (1 - ab - be - ca)S + 5" 
24 V + (l + a)(l + 6)(l + c) 



where 5 = | — m 2 . The fact that this quantity depends on the chemical potential ratios m, 
cii in a complicated way, meaning not linear, is quite puzzling. This is because one naturally 
expects the exponent to be linear in all chemical potentials (3, (3m, (3a,i, in which case their 
coefficients can be interpreted as zero point charges. For instance, in the 3d indices on S 2 x S , 
such an expected behavior perfectly showed up after a localization calculation [10]. As the 
expressions are not linear, it is not clear how one should interpret this quantity. It might have 
to do with some wrong regularization/renormalization that we have done. However, a curious 
fact is that (13. 9p will be precisely reproduced from the 5d calculus, as we shall explain. Note 
that, in the case with maximal SYM on S 5 , demanding the regulator to commute with all 16 
SUSY uniquely fixes the possible regulator to be e~^ E ~ Rl ^ for the index studied in [13]. As 
there are no more chemical potentials ratios one can introduce, the result does not depend on 
the continuous parameters of the theory. We shall discuss it more in section 4. 

The indices (13. 6p or (13.71) naturally admits a strong coupling expansion in 5d perspective if 
we interpret (3 as the 5d gauge coupling, as we expand the index in q = -C 1 for (3 ^> 1. On 
the other hand, to see the 5d gauge theory structures that we derived at weak coupling, one 
has to expand the index at /? <C 1. In general, given an expression which manifestly shows an 
index structure in q = e _/3 , performing a high temperature (3 expansion to all orders is difficult. 
However, when the expression manifestly takes the form of a Plethystic exponential, one can 
borrow the techniques used by Gopakumar-Vafa [2] which provides a conversion between strong 
and weak coupling expansions. [2] first proposed this when the index is given by the MacMahon 
function, but the same technique applies to our Abelian index. 

To perform the dual expansion easily, we start by noting that the letter index (13.61) can be 
rearranged to following form: 

-/3(l+c) e -/3(l+c)n _ g-^i+m-cJWj _ e -/3(i-m-c)\ 



e 



1 _ e -/3(l+c) (1 _ e -y3(l+a))(l _ e -/3(l+fe)^ 1 _ e -/3(l+c)) 

e -W+«> sinh sinh m-2 T -2c) 



1 - e-W+«0 2 sinh ^ sinh M sinh fl^ 



(3.10) 



Of course this decomposition obscures the manifest symmetry under a, b, c permutation, which 
shall be again made manifest after we finalize the weak-coupling expansion. The Plethystic 
exponential of the first term is proportional to the Dedekind eta function, whose weak coupling 
expansion is easily done by using its modular property. Calling this piece of index /(i), one 
obtains 



J (1) = e-^r/(e" /3(1+c) ) = 2 (1 + c )l e -^ + ^T 



exp 



"j8(1+c0 



p=i p 1 - e TUTc) 



(3.11) 
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where 77(g) = q 24 [^LiC 1 ~ 1 



n\-l 



q-2iPE[^ 



Now we explain how to dualize the Plethystic exponential of the second factor of (13.101) . 
which we call I^)- We write log I^) as 



E 



sinh *^+™- c > sinh *^-™- c > 



\ V 2 sinh 



P/3(l+a) 



sinh 



sinh 



E 



, . , s(^+m—c) . , s(i— m— c) 

as sinh —2— ^ sinh — 2— ^ 



s 2 sinh 



s(l+a) 



sinh 



s(l+c) 



sinh 



s(l+c) 



e /3 



(3.12) 

where e is a small positive number to regulate the s integral divergent around s = 0. To show 
this identity, one first uses sum over n on the right hand side before performing the integral, 
and uses the delta function identity 



E l TV i 71 s 



P 8(s -pfi) 



(3.13) 



p=— 00 



to obtain the left hand side. The sum over p is restricted to p > 1 as the integral variable s 
is positive, leading to the left hand side. Although each integral with given n is divergent as 
e — > 0, the divergences summed over n should cancel out as the left hand side is finite. This 
is the same manipulation as done in [2]. On the mathematics side, treatments similar to this 
have been developed after a new proof of Dedekind eta function's modular property in [4T] , 



On the right hand side of (I3.12p , we first separate out the piece in the integral which diverges 
at s — 0. We rearrange the right hand side as follows: 



eH 

1=— 00 

(\-4 



m 



s(l + a)(l + 6)(l + c) 
2 "°° ds 



sinh s(1 * + ™- c) sinh ^7~ c) 
2 sinh sinh ^ sinh 



e p 



m 



E "i 



'2 7T l I ! ± 

e e 



s(l + a)(l + 6)(l +c) ^ 
The second line is easily calculated by redoing the n summation first, which yields 



(3.14) 



7T" 



m 



6/3(l + a)(l + 6)(l + c) ' 



(3.15) 



As for the first line, we have sent e — > + to ease the analysis below. Even though each integrand 
at given n is finite so that one might think that the meaning of the integral is unambiguous 
at s — 0, this is not true after the infinite sum over n. Namely, had one been taken the 
summation first, it would have yielded a delta function X]^=-oo P^(s — p(3). With e > 0, we 
have considered the delta functions with p > 1 only, meaning that we understand the above 
expression as excluding the delta function contribution at s — 0. However, to use the contour 
integral technique later, it will be convenient to include this contribution and subtract it out. 
Understanding first line of (I3.14p this way, we have to subtract the contribution coming from 
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half of the /36(s) contribution at p = as follows: 



2 



(i - c) 2 - 



• i s(i+m— c) . , s(i— m— c) 

sinn 2 n sinn — — = 



s*(l + a)(l + 6)(1 + c) 2s sinh ^ sinh ^ sinh ^ 



J s=0 



/3 2a6c+(l 
24 _ 



ab — be — ca)5 + <5 2 



(l + a)(l + 6)(l + c) 



(3.16) 



Now with all the integrals on the first line of (13.14p including s = in the normal sense, can 
evaluate the integrals as follows. We consider the term with n = later, which is independent 
of p. The terms with are organized as 



v r — 



m 



sinh s{1 * + r c) sinh a( *T~ c) 



oo 

E 

n=l 



00 ds 

-oo S 



s(l + a)(l + 6)(1 + c) 2 sinh ^ sinh ^ sinh 

sinh S{ ^ + T~ C) sinh a( *T~ c) 



e p 



(3.17) 



(l" c ) 2 - 



s(l + a)(l + 6)(l + c) 2 sinh 



sCL+d) 



sinh 



g(l+j) 



sinh 



s(l+c) 



e 



We evaluate it by using contour integrals. We enclose the real axis of the s plane with a large 
semicircle on the upper half plane, as this semi-circle yields zero contribution. One obtains 



oo 1 



e 0(1+*) 
1 — e /3(i+ a ) 



-f(p,a,b, c) + 



e / 3 ( 1 + 6 ) 
1 — e w+io 



-/(p,6, c,a) + 



e w+c) 
— e /3(i+c) 



(/(P, c, a, 6) 



with 



f(p,a,b, c) 



pn(m—^+b) . pnlrn— ^+e) 

sin — r- 3 — - sin — 2 — - 



1+0 



1+a 



pTr(a-b) ■ pw(a—c) 

sin — n — i sin — 



(3.18) 



(3.19) 



1+a 1+a 

after some rearrangement. The three terms in the summation come from three classes of poles 
a t s — iq^>f+f> f+f w ith V > 1; respectively. The function f(p, a, 6, c) is symmetric under 
the exchange of its last two arguments b, c. Note that this function is the same as f(p, a, b, c) 
appearing in (12. 70ft while we were rewriting the perturbative partition function. This part now 
became manifestly invariant under a, b, c exchanges. This part also takes the form of instanton 
expansions at three fixed points. 

The integral with n = can also be evaluated in the same way. As the integrand with n = 
is an even function in s, we can integrate over — oo < s < oo and multiply \. Using the same 
contour, the result is 
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2 
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ds 
s 
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| — m z 



sinh 



s(§+m) 



sinh 



sU-m) 



8(1 + a)(l + b)(l + c) 2 sinh ^ sinh ^ sinh ^ 



^ /(p, a, 6, c) + /(p, 6, c, o) + /(p, c, a, 6) 



p=i 



2p 



2p 



2p 



(3.20) 
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This expression also formally looks invariant under a, b, c exchanges, but one has to be careful. 
As the last sum of — y is divergent, one cannot separate it out from the rest and claim that the 
first three summations are a, b, c symmetric. Actually, this expression is not a, b, c symmetric 
in a subtle way. To see why, it is again useful to recall that the above expression has the same 
form as the perturbative partition function that we rewrote in (12.691) . Considering the Abelian 
perturbative index, one finds that the exponential on the second line of (12.691) exactly takes 
the same form as our sum of residues ( 13.201) . If one multiplies = (1 + a) - 5(1 + 6)~2 for 
the Abelian theory to this exponential, one obtains the full perturbative determinant of the 
Abelian theory which should now be a, b, c symmetric. We thus identify 



cxp 



E 

, P =i 



f(p,a,b,c) f(p,b,c,a) f(p,c,a,b)-l 



(l + a)2(l + &)3 Z 



2p 2p 2p 

This finishes the evaluations of all ingredients in the weak-coupling expansion. 
Collecting all, (IXTTj) . (13TT5]) . (IBTToD . (l3~Tgj) . ( 13T2Tj) . one obtains 



c/(i) 

pert 



(3.21] 



A(i +a )(i +6 )(i +c ) 



-Pe - 



1+ 



2abc+(l-ab-ba-aa)(^-m 2 ) + (^-m 2 ) 2 
(l + a)(l + f>)(l + c) 



(^-c) 2 -m 2 
60 \ (l + a)(l + 6)(l + c) 



(3.22) 



xZp ert exp 



E 



P =i 



V 



_ 4n 2 p 
I — e /3(l+a) 



-f(p, m, a, b, c) + (a, b, c — > b, c, a) + (a, b, c — > c, a, b) 



We have included the Casimir energy factor e _/3e ° given by (13.91) . Although we already men- 
tioned after (13. 9 p that its interpretation is confusing to us, — (3e in the exponent exactly cancels 
another 0(f3) term on the fisrt line of (13.221) . Saying it differently, the 5d calculation is com- 
pletely reproducing eo, as we know that perturbative contribution does not have nontrivial ft 
dependence in 5d. All terms now look manifestly a, b, c symmetric, perhaps except for the 
(9(/3 _1 ) term, in the exponent of the first line. However, this is actually a, b, c symmetric. It 
may be rewritten, for instance, as 



7i~ 

6/3 



[\ — c) 2 — m 2 



7T 2 f-^+^ + m 2 



(3.23) 



+ c (l + a)(l + 6)(l + c)y 6/3 (l + a)(l + 6)(l + c) 

Actually, this is nothing but the high temperature leading form of the index ( 13.221) that one 
can easily derive by expanding the letter index (13. 6p at /3 <C 1. 

One can identify each factor as an appropriate contribution in the 5d calculation that we 
derived in section 2. The factor 



2tt 



(l + a)(l + 6)(l + c) 



(3.24) 



on the first line is the result of the integral over the single eigenvalue of A with the Gaussian 
measure (12.221) . as this variable does not have any other measure in the Abelian case. Z pcTt on 
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the second line is what we already identified as the perturbative determinant. The exponential 
on the second line looks like a product of instanton contributions from the three fixed points. 

Indeed, one can see that each factor is Z^ st with i = 1, 2, 3. For instance, let us just consider 
the last one: 



exp 



P =i p l-e wr+^ 



PE 



. 2 ( b-c , . 3(l + c)\ (b-c 3(l+c)" 
4tt z . 7r( -^-+m+— — '-) . K{— m +5 — - 

' i8(i+c) sin— — r- — -sm — 



l+c 



4tt 2 • 7r(c— a) ■ 7r(6— c) 

^ : Sin : : 
l+c l+c 



1 _ e ~/3(l + c) 



(3.25) 

Again the last PE is just a formal Plethystic, regarding e w+ c ) , y^, ^ as 'chemical potentials.' 

i+ 

2 



We rearranged the arguments in / to make the mass shift m + 5il±£) structure clear. 



(3) 

One can see that this factor agrees with Z^J Tt as follows. Firstly, one has to recall that the 

instanton partition function on IR 4 x S 1 has been calculated in an alternative method, from a 

related topological string partition function. On flat space, the object analogous to our Z^ t is 

the instanton partition function of the M = 2* theory with hypermultiplet mass m. This can 

be realized as M-theory compactified on a noncompact toric Calabi-Yau space to IR 4,1 . The 

topological string partition function on CY 3 calculates an index for the BPS states of the 5d 

theory [2]. This is exactly what the instanton partition functions are doing on M 4 x S 1 . So 

one finds two different expressions for a given partition function. In fact, for the Abelian 5d 

M = 2* theory, [12] uses some identities of topological vertices to recast the topological string 

partition function into a Plethystic form. The instanton part of the partition funciton is 

[ q sin(7r(6„ +m))sin(7r(6„ -m)) ] 

PE — — r , (3.26) 

l — q sm (— irei) sm [it 62) 

with e± = €1 ^ E2 ■ q is the fugacity for the instanton number and e±, €2 are the Omega deformation 
parameters, or chemical potentials for rotations on M 4 . The last ratio of 4 sine functions is the 
index for the center-of-mass supermultiplet for one superparticle moving on M 4 , preserving the 
SUSY of instantons. From this, and the expansion = q + q 2 + q 3 + • • • , this form of index 
was used in [7j to prove that the Abelian instantons on M 4,1 form unique thoreshold bound 
states at all instanton number, which is one of the key conjectures of M-theory pp. 

( 13.261) should also be the result one obtains from a direct instanton calculus, after summing 
over all the residues for poles given by Young diagrams (this time uncolored, as N = 1), weighted 
by q k for k instantons, similar to (12.481) but on the flat space. Recall that the differences between 
(12.481) and the residue formula on the flat space (written for instance in [7]) are firstly that the 
factors are absent in the arguments of sine's, and also that the mass shift m — > m + 3 ^ c ^ 
is absent. The statement that the U(l) instanton sum of the form 

k Y; \Y\=k 

equals (I3.26P has a good physical reason to believe, but has not been proved in full generality, 
as far as we are aware of. It has been checked in [7] explicitly up to k < 3. Now, accepting 
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this identity, it is a simple corollary that the Abelian instanton sum QZ3B)) with j^- c factor and 
mass shift enjoys a Plethystic-like form similar to (13. 26ft . The identification of parameters are 

_4=i, , 3(1 + c) . 

q ->■ e , ei a - c , e 2 -> 6 - c , m -» m + — - , (3.28) 

which precisely changes ( I3.26|) into (13. 25[) . This proves that the exponential on the second line 
of m is indeed Z&Z® zffi. 

The only remaining factor to be understood is the term of order (9(/3 _1 ) in the exponent 
on the first line. The term of order has no way to come out from the study of quantum 
fluctuation of dynamical fields on S 5 . However, to correctly probe the high temperature regime 
of a 6d index, such a factor is very natural as the index degeneracy often grows fast (although 
not as much as that of the ordinary partition function). In [13], it was argued that such terms 
with negative power in /3 should come from the constant shift of the S 5 action, which couples 
the parameters like g\ M ~ j3 in the theory to the background curvature of S 5 . Such constant 
couplings are never breaking the symmetries of SYM on S 5 . So as far as we can see at the 
moment, this seems to be a genuine ambiguity in our 5d approach. However, one is fitting just 
one ambiguous coefficient to a favorable value, after which the 6d physics of infinitely many 
BPS states is reproduced. In particular, this single fitting makes the whole partition function 
to take the form of an index at strong coupling. See section 4 for more details on the structure 
of such curvature couplings. Similar curvature couplings play important roles in [43], in which 
S-duality of 4d N = 4 SYM on curved spaces is discussed. To have S-duality, constant shifts 
to the action which takes exactly the same form as ours is assumed. 

Thus, up to the C(/3 _1 ) term which is ambiguous in 5d, our 5d calculation completely 
reproduces the known Abelian 6d index. 



3.2 Indices from 5d maximal SYM and AdSf duals 

We consider the partition function at a special subset of the parameter space /3, m, ei, e 2 , yielding 
an unrefined index in 6d. We set m — ±| (in the dimensionless unit after multiplying r) and 
ei = e 2 = 0. At these two points, one has maximal SYM on S 5 [13]. As the partition functions 
for the two cases with m = ±| turn out to be the same, we simply consider the case with 
m = \. At this point, a simple conjecture on the instanton part of the partition function was 
made in [13] for the U(N) gauge group. With this conjecture, the matrix integral including the 
perturbative part could be calculated exactly, yielding an index which survives nontrivial tests 
and also predicts an N 3 scaling of the (index version of) vacuum Casimir energy [13]. In this 
subsection, we derive the conjectured closed form expression for Z- inst for the U (N) theory, and 
then derive a similar closed form expression for SO(2N). We also propose a simple form for 
the instantonic correction for E n gauge group with maximal SUSY. We also elaborate on the 
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SU(N) 


SO(2N+l) 


Sp(N) 


SO(2N) 


Eg 


E 7 


E 8 


^4 


G 2 


r 


N-l 


N 


N 


N 


6 


7 


8 


4 


2 


\G\ 


N 2 -l 


N(2N+1) 


N(2N+1) 


N(2N-1) 


78 


133 


248 


52 


14 


c 2 


N 


2N-1 


N+l 


2N-2 


12 


18 


30 


9 


4 



Table 1: Group theoretic data 



6d index structure of the ADE partition function, including the perturbative part, and discuss 
the structure of the partition function for the BCFG gauge groups. The A n and D n partition 
functions completely agree with their gravity dual indices on AdSj x S 4 and AdSj x 5* 4 /Z 2 , 
respectively, at large N . 



We start from the perturbative part [13]. Although the partition function is generally a 
matrix integral with the factorized integrand consisting of Z pcrt and Z inst , in foresight we use 
the fact that Z- mst is independent of A with maximal SUSY. So in this case, the matrix integral 
can be done with the perturbative measure only. For gauge group G, one obtains |13^l 

z~=mh n H^V^'M^r^ 101 n 2sinh (^) . 

1 1 J aeA + V 7 V 7 aeA + 

(3.29) 

A + is the set of positive roots, r is the rank, c 2 is the dual Coxeter number, \G\ is the dimension 
of the semi-simple part of G, and p is the Weyl vector. The necessary group theory data are 
summarized in Table [TJ 

One can evaluate this perturbative determinant for various groups. We list the results for 
the classical groups. Firstly, for simply-laced group SU(N) (including the overall U{1) factor 
to make it U(N)) and SO(2N), one finds 

, . N N-l 

Z%£) = e §N(N*-l) (£_y Y[(l- e -^f-m (3>30) 

^ 7 m=l 

, „ v K N-l 

Z s p °[ 2N) = e f C ^ G I ( JL ) ft (1 - e-m^N-m (3 31) 

^ 7 m=l 

N-2 

X (l_ e -(JV-I)^[f] ] [ ^ 1 _ e -mP ){1 _ e -(2N-m-2)^\ 



m=l 



6 To deal with arbitrary gauge group in a unified way, we use the normalized trace which is independent 
of the representation. We define tr = ^ g ^ XR tiji. \6\ is the length of the long root, the Dynkin index xr of 



representation R is defined as tr R (T a T b ) = \0\ 2 x R 5 ab so that ti(T a T b ) = S ab . Here and below, we use the 
convention \9\ 2 = 2. We use the 5d action with the Yang-Mills term -j\ — trlF^F^"), using this trace. The 
unit instanton's action is 4n / j3 for all gauge groups with 27r/3 = g\ M . 



33 



For the non-simply-laced classical gauge groups, one obtains 

/ a \ f JV - 1 

Z™l 2N+1) = f J] (1 - e- m T~ m (3-32) 

^ ' m=l 

N-2 N 

X (l_ e -^)[f] Y[ [(l- e -(^)(l-e-^-m-i)^][— J "Q ^ _ e -f>{ m -^ 

m=l m=l 

z y) = e | C2|G| / _M _ e -!)iv- m (3 . 33) 

^ ' m=l 

JV-2 [ m+l l iV 

x(l- e -(^)[f] J] [(i_ e -(^)f )(1 _ e -(2iv- m )f^ » J "Q (!_ e -^ ) 



m=l m=l 



where [a] is the biggest integer no larger than a. Generally, the right hand side of (I3.29p can 
be written as 

r r 

Zpcn = (£\ 2 e &<»\G\+F>M JJ (i _ e -£(«-P)) = f M 2 e l c ^| J] (1 - e -««*>) , (3.34) 

where we used the Freudenthal-de Vries formula at the last step. 

Apart from the overall {j^)^ factor, the perturbative expression already takes the form 
of an index. For U(N) and SO(2N) cases in which we shall derive closed forms of instanton 
corrections, these factors obstructing index interpretation will be canceled against similar ob- 
structing factors from the instanton parts, making the whole S 5 partition function an index. It 
should be exciting to study whether similar cancelation happens for other gauge groups, espe- 
cially having in mind the possible interpretations of the non-simply-laced partition functions 
as twisted indices of 6d ADE theories. See the end of this subsection for comments. 

Note also that in all cases, the 'zero point energy' contribution all turns out to be — It 
was noted in [13] from the 5d perturbation theory viewpoint that this combination of numbers 
can be understood as f abc f abc = C2\G\ where f abc is the structure constant, which is the universal 
group theory factor for all 2-loop vacuum bubble diagrams. Note that the Casimir energy starts 
to appear from 0((3) ~ 0(gy M ), which is at 2-loop order. It is curious to find that the same 
combination Oi\G\ appears as an anomaly coefficient of the 6d ADE (2,0) theory |44j . 



As for the instanton part, we first consider the U(N) case. We can use the residue formula 
(12.481) for the third fixed point, and similar formulae for the other two fixed points by permuting 
a, b, c. Inserting the maximal SUSY values m — |, a, b, c — 0, one finds a very simple structure 
of these residues. Let us first stage our claim and then prove it. 

Inserting m = ~ in (I2.48P is simple. On the other hand, the limit 61,62 — > is somewhat 
tricky in that each saddle point contribution is not smooth at the point e\ = 62 = 0. (It may 
even diverge.) However, the total instanton partition function Z^ t Z^ t Z^ t is smooth at this 
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point, or at least believed to be so from general consideration as the partition function is finite 
on round S 5 . So we shall take this limit in a specific order which simplifies the analysis. We 
shall first take c — > 0, keeping a, b to be finite. We then take a = — b to zero. If we take the 
limit in this order, we claim that 

2 2 00 1 

41 Z£!L 1 and Z£L »?(e~~*T" = n , (3.35) 

n =i (1 -e~— ) N 

so that the net instanton contribution is Z inst = rj(e~^ r )~ N as conjectured in [X3] -0 Just to 
make sure, we also checked this claim in arbitrary possible order of the limit £1,62 — > 0, for 
N — 1, 2, 3 up to k < 3. To prove the claim, it suffices to show that Ziy u ... ,y N \ for all Young 
diagrams become 1 for the third fixed point, while Zy's at the other two fixed points become 
all (apart from the void case) in the limit. The last 0's obviously yield the total instanton 
partition function to be 1 at those fixed points. As for Z^ t , the instanton partition function 
with all Zry lt ... t Y N } = 1 is the generating function which counts the colored Young diagram with 

4tt 2 

e p being the fugacity of the box number k. This generating function is simply 77 . Note 
also that, as we shall be showing that all possible divergent terms in Zy cancel out to yield 
finite values within a fixed point (if the limit is taken in the above order), one can simply set 



at = in e w+ot) ->• e ? . 

We first prove that Zy's are identically at the first and second fixed points. We start by 
noting that all Zy's at the first fixed point \Z%\ = 1 contain at least one factor of 



(3.36) 



sIuttI 2 2 )sin7r( 2 1+a 2 I m= i sin ^L sini HL 



7r(a— 6) • 7r(a— c) ' 

i +a — " i +a sm -j^- sm - T ^- 



where we used a + b + c = and periodicity of sine function at various places. This basically 
comes from the fact that all instanton indices on M 4 x S 1 contain a factor of index from the 
centor-of-mass supermultiplet [T3] , which is ( I3.36P for the first fixed point. One can similarly 
show that Z's at the second fixed point has at least one factor like f)3.36p . with the role of a 
and b changed. Now take the limit c — > first, keeping a and b finite satisfying a + b = 0. 
Then obviously the above factor is zero from the sin factor in the numerator of (I3.36p . It 
is also easy to show that there are no other divergent factors in (12.481) as one takes this limit. 
So all Zy's from the first fixed point are zero, proving Z^ t = 1. One can also show Z-® t = 1 
in exactly the same way, proving half of our claim. 

Now let us consider the third fixed point in this limit. For Zy's at this fixed point, one 



7 As explained in [13] and section 3.1, the overall e ^ is an ambiguous factor from the 5d perspective. See 
section 4 for more discussions. 
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obtains the following at m — \ after a manipulation similar to (I3.36|) : 



Z{y u y 2 ,...,y N } = n n 



N ■ rTr X lj -(a-c)h i (s)+(b-c)v j (s)+b . A tj -(a-c)fc t (^)+(b-c)^(g)-a 

sin 7r 1+c sin n 1+c 



g j n 7r ^j-(a-c)fej(s)+(b-c)^(s)+(b-c) . Ajj -(a-c)fej (s)+(b-c)vj (s)-(a-c) 



»J=1 seY^ ^ " l+c ^ " l +c 

(3.37) 

Taking the limit c — )■ 0, keeping a = —6, one finds that the factor inside the product becomes 

sin7r(Ajj — ahi(s) + bvj(s) + b) ■ shi7r(Ajj — aht(s) + bvj(s) — a) ^ ^ 
sin7r(Aij — ahi(s) + bvj(s) + b) ■ sin7r(Ajj — ahi(s) + bvj(s) — a) 

47r 2 

So all Zy's at the third fixed point are 1, completing the proof of Z- mst — rj(e $ )~ N . 

We can extend the study of maximal SYM partition function for other classical gauge groups 
SO(N) and Sp(N). The instanton part of the SO(2N) gauge group at the maximal SUSY point 
can be analyzed in a similar manner. As a simple residue formula like (I2.48P is not available, 
one has to study directly the limit from the SO(2N) contour integral formula for the Sp(k) 
eigenvalues 0/. This analysis is explained in appendix A. 2. The result is 

Z°°r } = V(e-*T N (3.39) 
at the maiximal SUSY point m — |, e±, e-i — > 0. 

The instanton partition functions for other classical groups, SO(2N + 1) and Sp(N) could 
also be studied in this limit. These could be useful to develop and justify a twisted index 
interpretation of non-ADE partition functions, which we briefly explain at the end of this 
subsection. The analysis is more involved for Sp(N), as the instanton gauge symmetry is O(k) 
rather than SO(k), demanding us to consider various Z2 even and odd sectors. Also, the 
instanton indices for exceptional gauge groups are not available from our ADHM approach, 
due to the absence of such a formalism. However, from the simple result we obtained at the 
maximal SUSY point for U(N) and SO(2N) series, it is tempting to conjecture that 

2 /<-> \ n/2 

*^e--*-r=(f) V(e~r (3-40) 

for n = 6, 7, 8. Perhaps it may be possible to address this issue systematically, using recent 
developments in the exceptional instanton countings [15]. More generally, with general gauge 
groups, one might wonder whether 

Z^v(e-^r=( 2 4Y\(e-"Y (3-41) 



J 

is true. However, the analysis of Sp(N) instantons at k — 1 show that this formula cannot be 
generally true. See appendix A. 2 for the details. 
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p = 1 
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(1 I -I) 

\2' 2' 2' 


V2' 2^ 


b (fermionic constraint) 



Table 2: BPS Kaluza-Klein fields of AdS? x 5 4 supergravity, upon 5 4 KK reduction. At given 
p, the entries grouped by horizontal lines stay in the same representation of OSp(6, 2|4). 



To conclude with solid parts of our discussions only, we derived the following exact partition 
function for the U(N) and SO(2N) theories at the maximal SUSY point m = |, e! = e2 = 0: 



Z U(N) 



2 i SO{2N) 



iV(jV -1) 
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_ g—mp\m J. J. M 
' m=7V+l 

JV-2 



x (l_ e -(^-l)/5)[f] ] [ [(l_ e -^)(l. 



e mf3\N 



-(2N-m-2)f3 



(3.42) 



)] 



m=l 



which we interpret as the 6d index Tr [(— l) F e ^ E Rl ^] . The following large N indices can be 
obtained from the above exact indices: 



^U{co) 



2fSO(2oo) 
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(3.43) 



1 _ g2«-l)»(l - g 



2n\n 



L(i-?)(i-g 2 



where PP again denotes Plethystic exponential with the fugacity q = e 13 . 

One can also study these large N indices from their gravity duals on AdS-? x 5 4 and AdS-j x 
S' 4 /Z2. The large iV limit of the U(N) result was already shown to completely agree with the 
supergravity index on AdS-j x 5 4 [13J. Let us start by reviewing this proof. The BPS gravity 
fields on AdS-j after a KK reduction on 5 4 are given by Table The 50(6) charges denote the 
values of (ji, j 2 , J3) for its highest weight state, and 50(5) the values of (Pi, P 2 ) for its highest 
weight state. One has to sum over all their contributions to the index, weighting them by ± 
signs for bosons/fermions and also by q E ~ Rl . Multiplying to this the AdS-j BPS wavefunction 



factor 



one obtains the single particle gravity index in AdS-j x 5 4 . By reading off the 
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(?) 



information of Table [21 this was calculated in [13] to be 
1 
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oo p— 2 



p=3 n=l 



The multiparticle index obtained from I sp yields the MacMahon function 



I mp (q) = PE [I sp (q)} 



OO j 

n (i - o* 

n=l v y 



(1- 

(3.44) 



(3.45) 



5)" 



as the multiparticle gravity index on AdSj x S 4 . This completely agrees with the large N limit 
of U{N) index in (ET33) . 

Now we consider the gravity dual of the S0(2N) index. From Table [21 one has to appro- 
priately discard the Z 2 odd modes. As this is a parity or inversion action for the 'internal 
M 5 ,' it suffices for us to consider the representation of fields under S0(5). Firstly, all BPS 
bosonic fields on the first line of the table are chiral primaries. The states with 50(5) charges 
(Ri, i? 2 ) = (p, 0) can be viewed as being formed by rank p polynomials of two holomorphic co- 
ordinates of M. 5 . As the Z 2 parity acts as inversion on these coordinates, the Z 2 even fields are 
those with even p. Since the Z 2 orbifold preserves the whole OSp(8\A) symmetry, Z 2 even/odd 
natures of fields should be grouped into those of representations. Since the first four lines of 
Table |2] belong to the same representation at given p, one should only keep the fields with even 
p on AdS-j x 5* 4 /Z 2 . The last line of the table comes from a component of a constraint multiplet 
at p — 1 which saturates our BPS bound. As one can see from, say, table 3 in p. 31 of [12], 
this multiplet contains a state with E = 4, (^1,^2,^3) = (0,0,0) and (i?i,i? 2 ) = (1,0), which 
is Z 2 odd, meaning that this multiplet itself should be odd. Thus the last line of our Table 12] 
should be discarded. Therefore, the single particle gravity index on AdSj x S' 4 /Z 2 is given by 
the above AdSj x S* 4 summations with all p's restricted to even integers: 
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(3.46) 



In the above summation, we have set p 
N limit fl3Tl3|) of the SO (2 N) index. 



2m. This is again in precise agreement with the large 



Finally, let us briefly comment on the non-ADE partition functions that we obtained. Of 
course, the possible 6d index interpretation of the S 5 partition function critically depends on the 
choice of the theory: most importantly, the gauge group and matter content. Thus while ADE 
theories with one adjoint hypermultiplet are naturally expected to have index interpretations 
of 6d ADE theories, other theories may not. However, for non-ADE simple gauge groups, there 
could possibly be interpretations as 'twisted' indices of 6d ADE theories, where the twisting 
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is the action of outer automorphism of the 6d ADE gauge group o Following [46], one can 
consider a twisted circle compactification of the 6d (2, 0) theory on S 5 x S 1 . The twisting could 
be presumably inserting a discrete 'charge conjugation' operator to the 6d index, similar to 
the index discussed in [47]. Upon dimensional reduction to 5d, the zero modes of the twisting 
can be in adjoint representation of non-ADE 5d gauge groups. The Z 2 twisting of A 2N+ i (2, 0) 
theory is supposed to yield SO(2N+l) 5d SYM, Z 2 of A 2N and D N+1 yielding Sp(N) 5d SYM 
(with different values for the discrete theta angle), 1 2 of E% yielding 5d SYM, and finally 
Z 3 of D 4 yielding G 2 5d SYM. These are S-dual to the low dimensional gauge groups that one 
obtains by twisted reductions of higher dimensional Yang-Mills theories. See, for instance, [48] 
for a summary. So it would be interesting to see whether our non-ADE partition functions on 
S 5 can be quantitatively interpreted as such indices. 



3.3 Wilson loops and their large N gravity duals 

One can also consider more nontrivial BPS observables by inserting various operators on S 5 . 
For instance, one can insert BPS Wilson loops on S 5 . Wilson loops of 5d SYM have been 
studied in [49], related to the Wilson surface operators of the 6d (2, 0) theory. 

For simplicity, let us stick to the supercharge Q that was chosen in [I4J [161 E2] to localize 
the path integral. We also consider round S 5 only. Then the corresponding Killing spinor 
satisfies, among others, the properties 7 5 e = e and 7 3 e = — e in the notation of [13], where 
7 5 is the gamma matrix on S 5 with index on the Hopf fiber direction, and 7 3 is the internal 
gamma matrix along the direction of vector multiplet scalar <fi. Consider the following Wilson 
loop operator: 



W R = — — -tTR 
dim/t 

From the supersymmetry variation [13 



Pexp dstfApX 1 * + (f)\x\) 



(3.47) 



iSA 5 + ^= l - [et( 7 5 + f)A - At( 7 5 + 7 3 )e] , (3.48) 

the Wilson loop preserves Q when the curve x M (s) wraps the Hopf fiber, located at a point on 
CP 2 base. One can consider the localization of the path integral with Wr inserted. The analysis 
goes exactly in the same manner as the case without the insertion of Wr, except that we have 
to include the saddle point value of Wr inside the matrix integral. The scalar <p appears in this 
saddle point value: 

W R ->■ -tT R (e 2nX ) with A = r0 . (3.49) 

dim it 

One can of course consider the general case, but it is illustrating to consider the expectation 
value in the maximal SYM with m = \ and e% = e 2 = 0. To study the Wilson loop expectation 



5 We thank Yuji Tachikawa, and also Neil Lambert, Sung-Soo Kim for helpful comments on this. 
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value, one divides the path integral with Wr insertion by the partition function itself. Now, 
recall that the instanton contribution again is independent of the matrix A with maximal SUSY, 
so that they can be taken out of the integral. With the above normalization, the nonperturbative 
contributions exactly cancel out and it suffices to consider the perturbative contribution only. 
So one obtains the exact Wilson loop expectation value as follows: 

= ■ w\ I dX n ( 2 sinh ^) 2 tr * (e2 " A) e ~*" A2 ■ (350) 

This takes precisely the same form as the matrix integral for the Wilson loops in pure Chern- 
Simons theory on S 3 [501 [51] . upon relating our coupling with the Chern-Simons level k by 
analytic continuation (3 = — ^p. This is a continuation of the strange coincidence between the 
perturbative maximal SYM partition function on S 5 and pure Chern-Simons partition function 
on S 3 observed in [13], to the expectation values of BPS Wilson loops. 

No matter what the physical implication for this coincidence is, if there is anything at all, 
this relation is useful as the Chern-Simons partition function and Wilson loops are extensively 
studied in the literature. For instance, the expectation value of Wilson loops in fundamental 
representation of U (N) is given by [521 150] 

where we plugged in — ~ — » | to obtain the S 5 expectation value. The large N limit of this 
quantity is also interesting. One finds that 

eOMV I _ e -Nfi 
(W N ) s s = i-^- ^ . (3.52) 

The large N limit we take is not the 't Hooft limit of 5d SYM, since we keep fixed as we 
take N — > oo. This is the same as the large N limits we took for the partition functions in the 
previous subsection, as /3 is the chemical potential even in the large N AdSj duals. 

It would also be interesting to compare the above result with the gravity dual observable 
in AdSj x S A . The Wilson loop on S* 5 will naturally uplift to the Wilson surface operator 
on S 5 x S 1 , which winds the Hopf fiber of 5* 5 as well as wrapping the extra S 1 . As the 
fundamental Wilson loop in 5d gauge theory is roughly the locus of the fundamental string 
worldsheet ending on D4-branes, the Wilson surface is identified as the M2-branes ending on 
M5-branes. So one is led to consider the Euclidean M2-brane minimal worldvolume in AdSj 
[53| 154] with suitably compactified Euclidean time (preserving SUSY), which asymptotes to 
Hopf fiber x S l on boundary S 5 x S 1 . 

To calculate the M2-brane Euclidean action, one would first have to find the classical solution 
and then further carefully regularize the divergent action. However, it is easy to analyze the 
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large N scaling of this quantity from simple considerations. The M2-brane action is evaluated 
by studying 

S M 2 ~ t M i / \/detg (3.53) 

where g is the induced metric on the classical worldvolume solution £ 3 , and tmi ~ tt is the 
M2-brane tension related to the lid Planck length ip. As the classical configuration of E 3 is 
determined by its dynamics in AdSj, the only length scale which can affect the classical solution 
and the action is the AdSj radius i, at least when 0(/3) ~ 1 so that the compactification 
of Euclidean time does not provide another scale. So the only combination of dimensionful 
parameters which can appear is 

S M 2 ~ y ~ AT . (3.54) 

This is in agreement with the large N scaling of the Wilson loop expectation value f !3.52j) 
on S 5 . Comparison of the order 1 coefficients will be an interesting question. The naive 
compactification of the Euclidean time direction of global AdS? will presumably give the wrong 
answer, as this is not a supersymmetric compactification which is required from our index. If 
the M2-brane action calculated this way shows a mismatch in the 0(1) factor with f)3.52p . this 
probably will be a similar phonomenon as the discrepancy between the coeffcients of N 3 for 
the Casimir energy of AdS-j and the index version of it calculated in [T3J [T7] . 



4 Comments on ambiguities and maximal SUSY 

In the examples that we considered in sections 3.1 and 3.2, we encountered (small) ambiguities 
that we had to fix by hand, to have the 5d partition function to correctly reproduce the 6d 

A 

physics. They were factors of the form c 3 with positive constants A, which are supposed to 
provide the leading high temperature behaviors of the index free energies. From the structure of 
the 5d partition function, it seems that such terms cannot arise from the physics of dynamical 
fields on S 5 , as this is neither perturbative correction nor instanton correction, the latter taking 
the above form with negative A. Supposing that the coefficient above is suitably fixed, we were 
able to capture all degeneracy information for infinite towers of 6d BPS excitations. 

Physically, one may regard such a term as coming from the coupling of the parameters of the 
theory, such as gyui m in °ur case, with background geometry. This viewpoint was adopted 
in [13], motivated by a similar observation that 4d twisted M = 4 SYM on various curved 
manifolds exhibits S-duality only when the partition functions acquire nonzero contributions 
from similar curvature couplings [13] ■ For simplicity, let us consider the round S 5 for which all 
curvature invariants reduces to a polynomial of the scalar curvature. Also putting the maximal 
SYM there for simplicity, the only parameter g\ M has the dimension of length. So all possible 
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curvature couplings yield a constant shift to the action of the following form: 




in 



a_5 



/3 5 



+ 



a_ 3 



3 



+ 



+ + a 3 /? 3 + • • • • 



(4.1) 



Clearly, addition of such terms does not break any symmetry of the field theory. It seems that 
the first three terms cannot appear from the dynamics of QFT fields on S 5 . The remaining 
infinite series in positive powers of (3 takes the form of perturbative contribution to the parti- 
tion function. Addition of such a series with arbitrary coefficients of course will spoil all the 
perturbative considerations from the field theory. In fact, such additions will also spoil all 
perturbative studies of QFT on curved manifolds like S 3 or S 4 in the literature. So we only 
consider the first three terms when they seem to be inevitably necessary. 

The above terms in negative powers of /3 are naturally required from the 6d physics because 
we have larger degeneracies of states with growing energy. There are only finite number of 
coefficients that one can tune, and our strategy is fitting these small ambiguities to get the 
information of infinitely many BPS states. In the two examples (Abelian index and the index 
computed from 5d maximal SYM) that we concretely studied, the only nonzero coefficient was 
a_x. On the other hand, ^0- seems to be the right asymptotic scalings for the 6d BPS partition 
function (counting BPS bosons plus fermions) at high temperature because /3 -3 ~ T 3 would 
come from the three holomorphic BPS derivatives of the 6d theory. For this reason, we expect 
that a_5 should always be zero as the index cannot grow faster than the BPS partition function. 
It would be interesting to see in the general non- Abelian S 5 partition function whether nonzero 
a_3 is required, to make the full expression into an index. If that is the case, the implication 
is that the index would not be experiencing too much cancelation between boson/fermion so 
that there is a chance for the index degeneracy to be close to the true BPS degeneracy of the 
theory. With enough technical control over our general matrix integral and their strong-coupling 
expansion, whether a_3 ^ or not will be predictable from 5d considerations. 

When we generalize the above considerations to squashed S 5 , there are more quantities 
that one can construct from the background, other than the scalar curvature R ~ r~ 2 , so 
that the analysis could be much more complicated. Apart from various curvature invariants, 
we have already seen in section 2.1 that some background scalar and vector field had to be 
nonzero. It will be interesting to try to understand the structure of such terms from the off- 
shell supergravity methods [32], which were recently developed to conveniently construct QFT 
with rigid SUSY on curved spaces. One should first try to understand the (9(/3 _1 ) term of the 
Abelian index in section 3.1. 

Another source of possible ambiguity in our 5d calculation is that the index version of 
Casimir energy is somewhat difficult to interpret in general. If one can fully evaluate our general 
partition function and expand it at strong coupling, the result will give some values of this 
Casimir energy factor e _/3e °, supposing that the strong coupling expansion indeed takes the form 
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of index. Indeed, our Abelian partition function did provide the expression obtained directly 
in 6d in section 3.1. Also, (XT] computed the leading large N behavior of the perturbative free 
energy which takes the form (3eo with definite values of eo- p2] considered the case with nonzero 
(3, m. The Casimir energy here also exhibited the mass dependence eo oc (| — m 2 ) 2 which is 
hard for us to interpret. In any case, the fact that both 5d and 6d calculations yield the same 
nontrivial expression ( 13.91) is interesting. Perhaps, one might have to carefully reconsider the 
regulator dependence of the calculation of the S 5 partition function. 

Finally, as explained at the beginning of section 3.1, the fact that instanton moduli space has 
small instanton singularities could also be regarded as a possible ambiguity in our study. One 
may regard the 'natural' UV prescriptions such as [37] may be providing some UV completion 
beyond the potentially dangerous 5d QFT, at least in the BPS sector. 
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A Instanton countings for classical gauge groups 
A.l Indices and determinants for SO(N) 

SO(2N+l) case is a slight generalization of the SO(2N), at least at some formal level in index 
considerations. So we only discuss the index and partition fucntion for the SO(2N) gauge theory 
here. Using the results of [6], one can write down the SO(2N+l) results straightforwardly. 
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The fundamental characters for S0(2N) and Sp(k) are given as 



N 



(A.l) 



i=i 



i=l 



Since the adjoint characters for SO(2N) (Sp(k)) are obtained by (anti)symmetrization, we have 
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ch adjfc = ^( e ^+«^ + e #/-#7 + e -^/+^j + e -i4>i-i4>j) + ^( e 2^/ + e -^/) + fc _ ( A-3 ) 
/<j i=i 
Inserting them into f!2.38j) with slight modification [6], we can compute the equivariant indices 
for a vector multiplet and an adjoint hypermultiplet. Firstly, for the perturbative part, the 
indices are 
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for ei = a — c and e2 = & — c. The indices from instantonic contributions can be obtained in a 
similar way: 
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From these indices, we can read off the 1-loop determinant and the instanton partition function 
takes the form of a contour integration as follows. 
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Note that we have not divided it by the full dimension of Weyl group yet, thus part of gauge 
redundancy still remains. 

By choosing the proper contour, which is the same as U (N) case, the integration of (1A.7|) is 
nothing but extracting residues from poles that can be classified in terms of extended A-colored 
Young diagrams. For Zj- in SO(2N) theory, we consider k boxes arranged into 2N sectors; i.e., 
Y = (Y 1+ , Y 2+ , ... I Yi_, Y 2 _, . . . ), where Y i± denotes the Young diagram with boxes belonging 
to the i-t'th sector. For example, if </>/ = ±Aj — e + — ne\ — me 2 for nonnegative n,m, the J'th 
box is placed in the Yy- Young diagram. Due to the factor sin 7l - ^ I+ ^-^ 2e + ; which comes from 
ADHM constraints, here applies the following restriction: Both Yj + and Yj_ cannot be occupied 
at the same time. Collecting all residues with this rule, we can compute correctly. 

Now we encounter the remaining part of Weyl redundancy that connects Yj+ and Yj_ by 
mirroring. Hence we should mod out the set of mirrored configurations. This can be illustrated 
well by an example. Consider Z^ =2 for 50(4). All possible residues of extended 2-colored 
Young diagrams are collected, with division by number of redundancies, as follows. 

z 2 = |{(m, ■) + (•» -Im, ■)} + §{(-. m| •) + (■> -Km)} 
+H(B>-K •) + (•, -IB, -)} + §{(-,□!-, •) + (•. -h B)} 

+\{(n,n\-, ■) + (□,- 1 -,□) + (•,□!□, ■) + (•»□!•»□)}■ ( A - 8 ) 

For the case of higher N and k, Z& can be computed in a similar manner. 



2(j>I+m+ 3i±pl +e+ ^ 2 / -m-^4±£+£ ± ^ 2<t> I +m+^p±-e+ ^ ^ 2 <t> I -m-e+ 
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A. 2 Indices and determinants for Sp(N) 



The equivariant index for the perturbative part is obtained from that of U (N) theory by simply 
exchanging the U (N) character to Sp(N) character. The vector multiplet contribution is 
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and the index of the hypermultiplet in the adjoint representation is 
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For the instanton part, one has to carefully consider the dual gauge group of the instanton 
moduli space which is 0(k) gauge group at k instanton sector for Sp(N) gauge theory. The 
O(k) action is generated by two disjoint components : 0(k) + consists of the elements with 
determinant +1 and 0(k)- consists of the elements with determinant —1. The group action is 
generated by the following elements 

diag (e i<T2<t)1 , ■■■ , e ia2 ^ n ) for even k 
diag (e^ 1 , ■ ■ • , e ia2<l >", l) for odd k 



(A.11) 



for 0(k) + , and 



J<t>+ 



(A.12) 



diag (e iCT2 ^, • • ■ , e^ 2 *— 1 , 1, e^) for even k 
diag (e i<T2< ^, • • • , e iu2<t>n , e'§) for odd k 

for 0(/c)_, where o<i is Pauli matrix and p is the periodicity of <pi. (When we use the usual 
periodicity 0/ ~ 0/ + 2tt, the U(l) action e* P//2 becomes e tlT = —1. This corresponds to Z2 
quotient which is a member of O(k) group action. However, for our case, I used the periodicity 
p for <pj as the final formula has the terms like sin it ^{+' a ' an< ^ 4*1 * s 110 longer periodic under 2tt 
shift.) The index n is defined as k = 2n + x (x = or !)• 

Firsly, the equivariant indices of the instanton part for 0(k) + are 
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for the vector multiplet and 
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for the hypermultiplet in the adjoint representation. From the equivariant index, we read the 
contour integral formula for 0{k) + dual gauge group 
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Now we turn to the equivariant index for O(k)- action. Since the 0(fc)_ action behaves in 
very different way for even and odd k, we shall deal with two cases separately. Firstly, we write 
the equivariant index for odd k in the adjoint representation 
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and that of the vector multiplet and that of the hypermultiplet can be obtained from this index, 
which are given by 
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Then the final instanton index has the following contour integral form 
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Here we take the periodicity for (f>j as p = 



Secondly, the equivariant index for even k in the adjoint representation is given by 
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The vector multiplet index and the hypermultiplet index are then given by 
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Thus, the contour integral formula of the instanton index becomes 
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We can evaluate the above contour integrals for some small k. Especially, for k 
there is no integral variable and the result becomes simply 
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This becomes 1 as we take the limit a — > 0. In particular, this shows that the conjecture f)3.4ip 
cannot be generally correct. It might be interesting to see if the non-simply-laced nature of 
Sp(N) is causing complication. 
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A. 3 Instanton partition functions with maximal SUSY 



The main purpose of this subsection is to derive the S0(2N) instanton partition function at 
the maximal SUSY point. As a simple residue formula like ( 12.48(1 is unavailable, one should 
directly work with the contour integral expressions. As the argument for SO(2N) generalizes 
that for U (iV), let us first reconsider the derivation of the U(N) Z- mst with maximal SUSY using 
contour integral formula. 

The contour integrand for the U(N) k instantons at the third fixed point is: 

Sin7T t; — Sm7T r- — -n- Sin 71 ff 2 - Sin 71 ^tt— i — r S1H 7T TT^ Sin 71 — 2 

1+c 1+c I I 1+c 1+c I I 1+c 1+c 

</>/-Ai + 4? . 4>i-Xi— ^ 11 . <t>u+m+'§ . <j>u+m+'i;+3c 11 ^in ^ <t>lJ~ a+c ^ n ^ faj-b+c 

sm7T — 1+e 2 sm7T — jq^- 3 - i,j sin 7T — 1+e 2 sm7r j^y 2 - 7j j bm7r 1+c bm7r i +c 

(A.23) 

where we inserted e + = — y and e_ = yy at this fixed point. The prime in the second product 
means that sin7Tyy factor is absent when I = J. The residues at a pole will yield Zy, and 
from the first product, one sin factor in the denominator among sin-Tr ^'"^;^ = shiTT ^ 1 2 
is absent in the residue, as this is the location of the pole. Also, k — 1 of the denominators in 
the third product are absent as they are also the locations of the pole. 

Supposing that the factor in the product is neither containing the poles nor has an absent 
numerator (in the second product), one finds that most of the s ? ns ? n factors simplify at m — \ 
in one of the three limits: a — » 0, b — > or c — > 0. Firstly, one finds from the factor in the third 
product that 

sin 7r */ J +™-i+° gin ^U+rn-h+b sin n hj±L sin n hj±k 

> tt^t^- ( A -24) 

sm7r rij z S m7T yj : , sm tt , , sin7r y , , 

1+c 1+c 1+c 1+c 

when m — \. When we are taking the limit a or b — > 0, one finds that the corresponding residue 
is from one of this factor. This is because the factor with I = J (i.e. (pjj = 0) is included, 
and the numerator sin v"y^ sin ?r y^ implies that this residue is zero. To fully justify the last 
statement, one would also have to check that there are no other factors of the form smsm which 

' sin sin 

diverges in these limits, which turns out to be true. Exchaning the roles of a, b, c, this implies 
that Z[2t = Zj®. = in the c -> limit. 

Thus, we only need to consider the limit c — > limit of Z inst given by the above contour 
integrand. We consider various ratios of sine functions inside the products separately. We first 
consider the factors in the products which do contain neither poles in the denominators (for 
the first and third products) nor absent sine factors in the numerators (for the second product). 
The other cases are considered later separately. 

We first consider a factor in the first product when it does not contain a pole. This happens 
when for <pi and Aj which does not satisfy 0/ — A« ^ y. At m — |, the c — > limit can be 
obtained by just plugging in c = 0, as the denominator do not become zero in the limit at the 



49 



pole values of <f>i — Aj in this case. One obtains 

- Aj — 1) sin7r(0/ — Aj + 1] 



sm 7T 



1 . 



(A.25) 



2, supposing that I ^ J. Again 



sin 7t(0/ — Aj) sin iz{4>i — Aj) 

We then consider the c — > limit of the second factor at m 
one can just plug in c = 0, as the denominators remain finite in the limit at the pole values of 
4>ij. One obtains 

sin Trd) i / sin neb r j 

1 (A.26) 



sin + 2) sin ^{(pu + 2) 
for given I ^ J. We also consider the c — > limit for the factor in the third product, namely 
flA.24p . We first postpone considering k — 1 cases of /, J that yields poles. We also exclude the 
case labeled by J, / when the case with /, J contains a pole, as the latter will also turn out to 
be subtle in the c — > limit. In the other cases of J, J, again one can naivly plug in c = to 
obgain 

sin ttQjj + a) sin 7r(0/j - a) _ ^ ^ 

sin7r(0/j — a) sin7r(0/j + a) 
So far, all the trivial factors 1 is heading towards the proof that Zy — 1 at the third fixed point 
in this limit. 



To complete the proof, we should check the k factors containing the poles in the first and 
third products, and also the k terms with I = J in the second product. Firstly, the single factor 
containing the pole in the first product has shi7r(0 — Aj — y) = 0. So inserting (f>j — Aj 
the remaining three sin's, one obtains at m = \ 
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where the last expression is for small c. From the second product, there are k factors with 
I = J. At m = h one obtains 
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Finally, the k — 1 cases of J, J in the third product with poles, and the related cases k — 1 with 
J, /, are considered. Since one of the two sine factors in the denominator yields a pole, there 
are two possibilities between <j)u = a — c and 4>u = b — c. In the first case with (ftu = a — c, 
one obtains by combining the J, J factor and the J, I factor the following: 

. c— a+b 
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l+c " l+c 0111 " i +c " l+c 

before taking any limit. In the second case with (ftu = b — c one similarly obtains 

-26 
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(A.31; 
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Their c — > limits become 



i )u = a — c poles : — r , (f>u = b — c poles : 



3 cos(7ra) 3 cos(7rfe) 

For these factors, one further has to take the a = — b — > limit to obtain the final expression. 
In both cases, one obtains y. So combining all, one obtains 



2vrc / 3 \ k /2ttc x k 



2txc 



-l) fe . (A.32) 



The last (— l) fe is simply canceled with similar sign factors one has to include in the <fii integra- 
tion measure jl], which leads to f)2.48j) . This completes the proof of Z inst = r/~ N without using 
the handy residue formula ( 12.481) . 

For SO(2N), apart from the measure of the above form, resembling that for U(2N) with 
restricted VEV Aj in SO(2N) Cartan, one finds the following extra measure for SO(2N) at the 
third fixed point: 

sin7r^^sin7r ^ + 1 l J+3c sin7r^±^sin7r ^+l J " 3c 

1+c 1+c w 1+c 1+c 
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t-t sinTr^sinvr^^ sin tt^ sin 

xTT i±| ¥°— 3 x i±| ¥°— a . (A.33) 

11 . 2^+m+l . 2<>+m+|+3c . 2</>/-m-§ . 2(4-m-|-3c v 7 

J sm7r — — — - sm7T tt^ — sm7r — r- — — sin 7T 2 — 

1+c 1+c 1+c 1+c 

The c — > limits of six s . ns ? n factors at m = ~ can be attained by simply plugging in c = 0, 
which yields 



sin no sin Tim sin n0 sin 7r< 



shi7r(0 + 2) sin7r(0 + 2) ' shi7r(0 — 2) sin7r( 
for the two factors on the first line, 



(A.34) 



sin7r(0 + b) sin7r(</> + c) sin7r(</> — 6) sin7r(</> — c) ^ (A 35) 

sin 7r(0 — 6) sin 7r(0 — c) sin 7r(0 + 6) sin ti{4> + c ) 

for the two on the second line, and 

sin(27T0) sin(27T0) sin(27T0) sin(27T0) 



sin7r(20 + 2)sin7r(20 + 2) ' sinvr(20 - 2) shi7r(20 - 2) 
for the two on the last line. 



(A.36) 



The above proves that Zy + y_ = 1 for the third fixed point with the pair of iV-colored 
Young diagrams labeling the poles, when we take the go to the maximal SUSY point by taking 
c — > first. To finally understand Z inst with maximal SUSY, one has to understand the allowed 
pair (Y + ,Y_) of colored Young diagrams which generally provide nonzero residues. One can 
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first check that the pair Y± = (Y\±,Y2±,--- ,Yn±) has zero residue when both Yi + , Yi_ are 
nonempty with a given i, 1, 2, ■ • • , A. Let us check this by considering the case with both Yi± 
being nonempty. In this case, the <fi values for the first boxes of the two Young diagrams Yi+, 
Yi_ are (pj = A, + y and 4>j = — Aj + y, respectively Then, consider the second factor on the 
first line of ( 1A.33|) . This factor contains sinTr^-y^y^ = in its numerator. This proves that 
there are no poles when both Y{ + , Yj_ are nonempty. 

So Z- mst = Z inst with maximal SUSY is the generating function for the pair of A-colored 
Young diagrams satisfying the above constraint, which counts the diagrams taking the gauge 
symmetries of this system into account. The relevant gauge symmetry here is the Weyl group 
of 0(2N), when its A Cartans Aj are nonzero and all different genericallyj§ The Weyl group of 
0(2N) has 2^ A! elements. As AM of them simply permutes the A eigenvalues Aj, this part of 
the symmetry is broken with all Aj's being different. The remainig 2 N elements with a given 
set of Aj's act by flipping signs of these eigenvalues in all possible ways. Note that under the 
change Aj — > — Aj, the two diagrams Yj± are exchanged. Since we just checked that one of yi± is 
always empty for any i, we can use these 2 N Weyl transformation to move all nonempty Young 
diagrams to Y + . So the problem boils down to the counting of one set of A-colored Young 
diagrams. Just like the U(N) case, its generating function is given by 

ZST = »(«-*)-" ■ (A-37) 
which proves the claim made in section 3.2. 



B Off-shell action from supergravity 

In this section, we derive the classical action ( 12. 22ft at the saddle point with a constant scalar 
in the vector multiplet. We need some information about the off-shell action on squashed S 5 . 
The simplest way is to use the off-shell supergravity method [32]. Off-shell formalism of 5d 
supergravity is known in [55j EE] • 

Let us first summarize our strategy. From the dimensional reduction of twisted product 
space S 5 x S l , we identified the squashed S 5 geometry as well as other background scalar and 
vector fields. They are to be regarded as nonzero background fields in the 5d supergravity 
coupled to our matter vector multiplet, containing <p and the gauge field A^. We can also 
reduce the canonical Kiling spinor equation on twisted S 5 x S 1 to 5d. This has to be identified 
with the vanishing condition of gravitino SUSY variation. By comparing the two, one can 
decide the values for various auxiliary fields in the off-shell supergravity. With these fields 

9 Note that it is more natural to regard the gauge group as 0(2N) rather than SO(2N), as one motivation 
for the existence of this type of 6d (2, 0) theory comes from M5-branes at an orbifold. Brane pictures are giving 
O type rather than SO type gauge groups. 
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determined, one studies SUSY variations for other fermions to fix the values of more bosonic 
auxiliary fields that we need to compute our off-shell field theory action. Finally, plugging in 
the SUSY configuration of our matter vector multiplet (corresponding to our saddle points in 
localization calculus), we shall obtain (I2.22p . 

In 6d, the chemical potentials in the index leads to the following shifts of the Euclidean 
time translation on 5 5 x 1 (we set the S 5 radius r = 1 in this section, for simplicity): 

V T -> V T + a 3l + bj 2 + c j3 - Rl+ 2 2 -m(R 1 -R 2 ) . (B. 1) 

The Killing spinors surviving the compactification of time direction, and thus the 5d reduction, 
are constant in r. In the time component of Killing spinor equation, 

R -\- R \ 1 

d T + aji + bj 2 + cj 3 ^— ^ - m{R l - R 2 ) J e± = =f-e± , (B.2) 

one seeks for constant solutions with generic values of a, b, c (a + b + c = 0) and m. This leads 
us to 2 real components obeying the following conditions: 

Jie± = J 2 e± = J3e± = e± = T~e± . (B.3) 

They are Q, S which annihilate our BPS states in the index. 
The metric on S 5 x S 1 with twist (12 .51) can be written as 

dsl = dsl + a" 2 (dr + C) 2 (B.4) 

with 

a -2 = (1 — n 2 a 2 ) , C = a 2 irn 2 aid(j)i . (B.5) 

and a should be identified as the vector and scalar in a background vector multiplet on the 
squashed S 5 in off-shell supergravity. As the chemical potentials for the S 5 angular momenta 
are explicitly encoded in the geometry, the twistings in the time derivative are now restricted 
only to the internal part: 

d T + a 3l + bj 2 + cj 3 - ?±±^1 - m(i? 1 - it> 2 )) (d T - ?±±^ _ m ( J R 1 _ R 2 )\ ( B .6) 

It is the last expression which replaces all time derivatives in the SUSY and Abelian equation 
of motions in 6d. When we reduce 6d Abelian fields to squashed S 5 , we only keep the r 
independent modes. Thus, all twisted time derivatives are essentially replaced as 

d T ^-( Rl + R2 +m(R 1 -R 2 )) . (B.7) 



Let us define V = dC to be the field strength of C. We also choose the vielbein as 

e a , e 6 = a' 1 (dr + C) (B.8) 
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where a = 1,2,3,4,5 and 6 are the 6d indices in the local Lorentz frame. Then starting from 
the Killing spinor equation of twisted S 5 x S 1 and reducing to 5d following the rules above, the 
5d part of the Killing spinor equation can be rewritten as 

(D a - ^V bc la b )j e± = laV± , V± = i (~a Rl+ 2 R2 - -^W 6 + ^ a a^ e ± (B.9) 

This can be compared with the vanishing of the gravitino SUSY variation 

<% = 2V + X -v ah liiah e - lv n = (B.IO) 

in 5d, where V^e 1 = (d^ + \oj^ a bl ab + §fyx) e % — KAe^. For the two equations to be compatible, 
we set some 5d supergravity fields as 

^ = 0, V„ = -cJ Rl + R A , v = --^V, e = e ±J V = V± (B.ll) 

The first equation for is a gauge-fixing. The vanishing of the gaugino SUSY variation in the 
Weyl multiplet, together with the \x = 6 component of the S 5 x S 1 Killing spinor equation, also 
determines an auxiliary field D. It appears that the solution is as simple as D = — -^iV 2 . 

One can also consider the SUSY of off-shell vector multiplets. Let / label the vector multi- 
plets. In the bosonic sector, each vector multiplet consists of a gauge field A 1 , a real scalar 7 , 
and an SU(2) triplet of auxiliary fields {D 1 ) 1 ■ with i,j = 1, 2. In our problem, we need two sets 
of vector multiplets. One is the vector multiplet fields which are dynamical in our QFT, which 
can be labeled by many / components if we are considering non-Abelian generalization (with 
gaugings in supergravity). We need an extra background vector multiplet, which we label by 
1 = 0, whose vector and scalar components are the background 'RR-field' and 'dilaton' a 
that we identified after the reduction of twisted S 5 x S 1 on the circle. In fact, the above back- 
ground values are invariant under the off-shell SUSY transformation for the vector multiplet, 
provided that we suitably turn on the D-term auxiliary fields: 

V° = (A°, 0°, D ) = (C M) a, -wV 3 ) . (B.12) 

Nonzero value of D°, with a choice a 3 , breaks SU{2) R-symmetry in the curved background 

P3EEH]. 

Now we consider the bosonic part of the off-shell action which contain our dynamical vector 
multiplet fields. As the main purpose of this appendix is to identify the correct saddle point 
action, we do not need to consider our hypermultiplet here. We take the cubic polynomial in 
the supergravity to be 

M = CukVW = \a<p 2 + • • • , (B.13) 

where • • • denote the possible terms which do not contain 0. As these do not affect our QFT 
Lagrangian on curved space, we do not pay attention to them. The first term is normalized 
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to have the canonical kinetic term for the vector multiplet fields on S 5 with a — 1. Then the 
vector multiplet part of the bosonic action can be read off from the off-shell supergravity action 
with above A/", which is 

= \ ( 16^ + \ R + 16^) <# ~ 4> 2 " 

-20 ^V ab F ab - l -d a aD a( f> + Z -a 2 (a%D^ (B.14) 

-a (~F ab F ab - ^D a cf>D a <t> - \d i3 dA + e^-e^C^F^ . 

The auxiliary fields remaining in the above expression is determined above. So this yields 
the vector multiplet part of the bosonic action with our dynamical field A^, and the triplet 
auxiliary field D l y 

Now we finally come to the classical action at the saddle point. Note first that that the off- 
shell SUSY transformation for all vector multiplets take the same form. So even our dynamical 
vector multiplet fields can have SUSY configuration proportional to the value (1B.12j) for the 
background vector multiplet. Multiplying a constant O valued in the gauge algebra, one obtains 

V 1 = (A^, 0, D) = O V° = (0 o C M ,0 o a,0oHaV)) . (B.15) 

0o labels our off-shell saddle point on the squashed S 5 , which also correctly reduces to the 
known saaddle point values of the round S 5 studied previously [HJ [161 US] 5 U P to different 
definitions of D. Note that similar configuration was studied in [18] for a different class of 
squashed S 5 . Plugging in all auxiliary fields and our saddle point configuration, one obtains 



^9YM e A 



a 3 (^R + J^V^J + a 3 d a ad a a + 3« 5 + '-e^e^C.V^V^ 



% (B.16) 



After evaluating the integral over the functions appearing in the square bracket, one obtains 

° J V9 d ^°"^ M (l + a 1 )(l + a 2 )(l + a 3 ) 0(1 + a)(l + 6)(1 + c) ' 
where 0o = A in our setting with r = 1. This leads to ( 12.22ft . 
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